THE DYNAMIC GRANULARITY MEMORY SYSTEM
|
|
|
- Peregrine Stephens
- 5 years ago
- Views:
Transcription
1 THE DYNAMIC GRANULARITY MEMORY SYSTEM Doe Hyun Yoon IIL, HP Labs Michael Sullivan Min Kyu Jeong Mattan Erez ECE, UT Austin
2 MEMORY ACCESS GRANULARITY The size of block for accessing main memory Often, equal to last-level cache line size Modern systems use coarse-grained (CG) memory access 64B or larger Amortize control & ECC overhead Prefetching 2
![a[ b[i] ] += x; } Buffer a](/docs-images/90/102419259/images/3-3.jpg "Initialized with random numbers Waste")
3 CG ACCESS MAY WASTE BW GUPS microbenchmark for( i=0; i<n; i++ ) { a[ b[i] ] += x; } Buffer a Initialized with random numbers Waste BW on unused data 3
4 CAN WE WASTE BW? CG access often improves performance Large cache lines reduce miss rate due to prefetching Off-chip BW doesn t scale with # cores Power is the limiting factor We shouldn t waste the finite off-chip BW 4
5 HOW TO EFFICIENTLY UTILIZE OFF-CHIP BW? Prior work: AGMS [ISCA 11] Combine CG and FG accesses Need SW help for ECC support DGMS Source code, compiler, OS, virtual memory, HW-only variant of AGMS Truly dynamic granularity adaptation 5
6 ADAPTIVE GRANULARITY MEMORY SYSTEM [ISCA 11] 6
7 AGMS [ISCA 11] Combine coarse-grained (CG) and fine-grained (FG) accesses CG for high spatial locality regions FG for low spatial locality regions Higher throughput Lower DRAM power 7
8 Reg/Demux SUB-RANKED DRAM MODULE Independently control individual DRAM chips Access granularity = 8bit x burst 8 = 8B DBUS ABUS x8 x8 x8 x8 x8 x8 x8 x8 x8 SR0 SR1 SR2 SR3 SR4 SR5 SR6 SR7 SR8 8
9 Reg/Demux 64-bit data + 8-bit ECC (SEC-DED) B0 B1 B2 B3 B4 B5 B6 B7 E 0-7 B8 B9 B10 B11 B12 B13 B14 B15 E 8-15 B16 B17 B18 B19 B20 B21 B22 B23 E Burst 8 B24 B32 B25 B33 B26 B34 B27 B35 B28 B36 B29 B37 B30 B38 B31 B39 E E B40 B41 B42 B43 B44 B45 B46 B47 E B48 B49 B50 B51 B52 B53 B54 B55 E B56 B57 B58 B59 B60 B61 B62 B63 E ABUS x8 x8 x8 x8 x8 x8 x8 x8 x8 SR0 SR1 SR2 SR3 SR4 SR5 SR6 SR7 SR8 9
10 Reg/Demux 8-bit data + 5-bit SEC-DED or 8-bit DEC B0 E0 B8 E8 B16 E16 B24 E24 B1 E1 B9 E9 B17 E17 B25 E25 B2 E2 B10 E10 B18 E18 B26 E26 Burst 8 B3 B4 E3 E4 B11 B12 E11 E12 B19 B20 E19 E20 B27 B28 E27 E28 B5 E5 B13 E13 B21 E21 B29 E29 B6 E6 B14 E14 B22 E22 B30 E30 B7 E7 B15 E15 B23 E23 B31 E31 ABUS x8 x8 x8 x8 x8 x8 x8 x8 x8 SR0 SR1 SR2 SR3 SR4 SR5 SR6 SR7 SR8 10
11 SOFTWARE SUPPORT IN AGMS Different data/ecc layouts for CG & FG Requires software help Extend virtual memory interface OS/runtime manages CG&FG pages Programmer/compiler annotates preferred granularity Need to change every level of system hierarchy! 11
12 DYNAMIC GRANULARITY MEMORY SYSTEM 12
13 DGMS Unified data/ecc layout for CG & FG No SW support HW-only variant of AGMS Comparable or better performance Easier to implement Challenge: How to predict access granularity dynamically? 13
14 UNIFIED DATA/ECC LAYOUT 64-bit data 8-bit ECC (SEC-DED) B0 B8 B16 B24 B32 B40 B48 B56 E 0-7 B1 B9 B17 B25 B33 B41 B49 B57 E 8-15 Burst 8 B2 B3 B4 B10 B11 B12 B18 B19 B20 B26 B27 B28 B34 B35 B36 B42 B43 B44 B50 B51 B52 B58 B59 B60 E E E B5 B13 B21 B29 B37 B45 B53 B61 E B6 B14 B22 B30 B38 B46 B54 B62 E B7 B15 B23 B31 B39 B47 B55 B63 E
15 Access the whole 72B CG ACCESS B0 B8 B16 B24 B32 B40 B48 B56 E 0-7 B1 B9 B17 B25 B33 B41 B49 B57 E 8-15 Burst 8 B2 B3 B4 B10 B11 B12 B18 B19 B20 B26 B27 B28 B34 B35 B36 B42 B43 B44 B50 B51 B52 B58 B59 B60 E E E B5 B13 B21 B29 B37 B45 B53 B61 E B6 B14 B22 B30 B38 B46 B54 B62 E B7 B15 B23 B31 B39 B47 B55 B63 E
16 FG ACCESS Access 8B data and 8B ECC B0 B8 B16 B24 B32 B40 B48 B56 E 0-7 B1 B9 B17 B25 B33 B41 B49 B57 E 8-15 Burst 8 B2 B3 B4 B10 B11 B12 B18 B19 B20 B26 B27 B28 B34 B35 B36 B42 B43 B44 B50 B51 B52 B58 B59 B60 E E E B5 B13 B21 B29 B37 B45 B53 B61 E B6 B14 B22 B30 B38 B46 B54 B62 E B7 B15 B23 B31 B39 B47 B55 B63 E
17 AVOIDING CONTENTION ON ECC DRAM 8 B 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B ECC 8 B 8 B 8 B 8 B 8 B 8 B 8 B 8 B SR 0 SR 1 SR 2 SR 3 SR 4 SR 5 SR 6 SR 7 SR 8 17
18 DGMS DESIGN Core 0 Core 1 Core N-1 Sector cache $D L2 $I $D L2 $I $D L2 $I Last Level Cache Memory Controller Sub-ranked memory w/ unified data/ecc layout DRAM 18
![GRANULARITY PREDICTION Spatial Pattern Predictor [Chen; HPCA 04] SPP $D Core 0 L2 $I SPP $D Core 1 L2 $I SPP $D](/docs-images/90/102419259/images/19-2.jpg "Core N-1 L2 $I Last Level Cache Memory Controller Sub-ranked Memory Which words within a cache line will be used?")
19 GRANULARITY PREDICTION Spatial Pattern Predictor [Chen; HPCA 04] SPP $D Core 0 L2 $I SPP $D Core 1 L2 $I SPP $D Core N-1 L2 $I Last Level Cache Memory Controller Sub-ranked Memory Which words within a cache line will be used? 19
20 SPATIAL PATTERN PREDICTOR [CHEN; HPCA 04] Load/Store Update CPT PC + DA Tag Status Data Used Idx Status Idx Pattern L1 Data Cache T CPT Default Evicted or Subsector miss PHT Request To L2 F PHT hit 20
21 SPP ACCURACY 1.0 Not predicted, but Referenced Predicted & Referenced SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN streamcluster s-cluster stream 21
22 Case 1) SPP LIMITATIONS Application accesses 5~7 words per cache line Case 2) App1 has low spatial locality, MPKI is 1 App2 has high spatial locality, MPKI is 20 Minimizing traffic doesn t always improve performance 22
23 PREDICTION CONTROLLER Core 0 Core 1 Core N-1 SPP $D $I SPP $D $I SPP $D $I LPC L2 LPC L2 LPC L2 Ignore SPP if AvgRefWord > 3.75, or if row-buffer hirate > 0.8 GPC Last Level Cache Memory Controller Treat all requests are CG if CG requests are dominant (more than 80% of MC queue) Sub-ranked Memory LPC & GPC prevent performance degradation in some CG-friendly apps 23
24 EVALUATION 24
25 Zesto simulator EVALUATION 8 out-of-order x86 cores Private caches: 32kB I/D L1, 256kB unified L2 Shared last-level cache: 8MB DrSim: detailed DDR3 DRAM model Memory intensive multiprogrammed workloads 25
26 Weighted Speedup 6 SYSTEM THROUGHPUT CG SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 26
27 Weighted Speedup 6 SYSTEM THROUGHPUT CG AGMS SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 27
28 Weighted Speedup 6 SYSTEM THROUGHPUT CG AGMS DGMS SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 28
29 Weighted Speedup LOW SPATIAL LOCALITY APPS CG AGMS DGMS SSCA2 canneal em3d mst gups mcf omnetpp 29
30 Weighted Speedup 6 SYSTEM THROUGHPUT CG AGMS DGMS SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 30
31 Weighted Speedup HIGH SPATIAL LOCALITY APPS 6 CG AGMS DGMS omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 31
32 Weighted Speedup 6 SYSTEM THROUGHPUT CG AGMS DGMS SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 32
33 Weighted Speedup MIXED CASES MIX1: SSCA2 x2, mst x2, 6 em3d x2, canneal x2 MIX2: SSCA2 x2, canneal x2, mcf x2, OCEAN x2 MIX3: canneal x2, mcf x2, bzip2 x2, hmmer x2 MIX4: mcf x4, omnetpp x4 MIX5: SSCA2 x2, canneal x2, mcf x2, streamcluster x2 4 3 CG AGMS DGMS r stream MIX1 MIX2 MIX3 MIX4 MIX5 33
34 Normalized Throughput/Power POWER EFFICIENCY CG AGMS DGMS SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 34
35 Weighted Speedup SYSTEM THROUGHPUT (NO ECC) 7 6 CG AGMS DGMS SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 SSCA2 canneal em3d mst gups mcf omnetpp lbm OCEAN s-cluster stream MIX1 MIX2 MIX3 MIX4 MIX5 35
36 CONCLUSIONS Dynamic Granularity Memory System HW-only variant of AGMS Truly dynamic granularity adaptation Higher performance [31% vs. CG] Lower DRAM power [13% vs. CG] More in the paper Reg/demux and address/command bus bandwidth LPC&GPC details DGMS with chipkill-correct support 36
37 THE DYNAMIC GRANULARITY MEMORY SYSTEM Doe Hyun Yoon IIL, HP Labs Michael Sullivan Min Kyu Jeong Mattan Erez ECE, UT Austin
Virtualized ECC: Flexible Reliability in Memory Systems
Virtualized ECC: Flexible Reliability in Memory Systems Doe Hyun Yoon Advisor: Mattan Erez Electrical and Computer Engineering The University of Texas at Austin Motivation Reliability concerns are growing
Balancing DRAM Locality and Parallelism in Shared Memory CMP Systems
Balancing DRAM Locality and Parallelism in Shared Memory CMP Systems Min Kyu Jeong, Doe Hyun Yoon^, Dam Sunwoo*, Michael Sullivan, Ikhwan Lee, and Mattan Erez The University of Texas at Austin Hewlett-Packard
Virtualized and Flexible ECC for Main Memory
Virtualized and Flexible ECC for Main Memory Doe Hyun Yoon and Mattan Erez Dept. Electrical and Computer Engineering The University of Texas at Austin ASPLOS 2010 1 Memory Error Protection Applying ECC
Flexible Cache Error Protection using an ECC FIFO
Flexible Cache Error Protection using an ECC FIFO Doe Hyun Yoon and Mattan Erez Dept Electrical and Computer Engineering The University of Texas at Austin 1 ECC FIFO Goal: to reduce on-chip ECC overhead
Memory Mapped ECC Low-Cost Error Protection for Last Level Caches. Doe Hyun Yoon Mattan Erez
Memory Mapped ECC Low-Cost Error Protection for Last Level Caches Doe Hyun Yoon Mattan Erez 1-Slide Summary Reliability issues in caches Increasing soft error rate (SER) Cost increases with error protection
Robust GPU Architectures Improving Irregular Execution on Architectures Tuned for Regularity
Robust GPU Architectures Improving Irregular Execution on Architectures Tuned for Regularity Mattan Erez The University of Texas at Austin (C) Mattan Erez 2 Lots of interesting multi-level projects Resilience/
CLEAN-ECC: High Reliability ECC for Adaptive Granularity Memory System
-ECC: High Reliability ECC for Adaptive Granularity Memory System Seong-Lyong Gong ECE, UT Austin sl.gong@utexas.edu Jinsuk Chung ECE, UT Austin chungdna@gmail.com Minsoo Rhu NVIDIA mrhu@nvidia.com Mattan
Exploring Latency-Power Tradeoffs in Deep Nonvolatile Memory Hierarchies
Exploring Latency-Power Tradeoffs in Deep Nonvolatile Memory Hierarchies Doe Hyun Yoon, Tobin Gonzalez, Parthasarathy Ranganathan, and Robert S. Schreiber Intelligent Infrastructure Lab (IIL), Hewlett-Packard
Let Software Decide: Matching Application Diversity with One- Size-Fits-All Memory
Let Software Decide: Matching Application Diversity with One- Size-Fits-All Memory Mattan Erez The University of Teas at Austin 2010 Workshop on Architecting Memory Systems March 1, 2010 iggest Problems
Lecture 5: Scheduling and Reliability. Topics: scheduling policies, handling DRAM errors
Lecture 5: Scheduling and Reliability Topics: scheduling policies, handling DRAM errors 1 PAR-BS Mutlu and Moscibroda, ISCA 08 A batch of requests (per bank) is formed: each thread can only contribute
Virtualized and Flexible ECC for Main Memory
Virtualized and Flexible ECC for Main Memory Doe Hyun Yoon Electrical and Computer Engineering Department The University of Texas at Austin doehyun.yoon@gmail.com Mattan Erez Electrical and Computer Engineering
Balancing DRAM Locality and Parallelism in Shared Memory CMP Systems
Balancing DRAM Locality and Parallelism in Shared Memory CMP Systems Min Kyu Jeong *, Doe Hyun Yoon, Dam Sunwoo, Michael Sullivan *, Ikhwan Lee *, and Mattan Erez * * Dept. of Electrical and Computer Engineering,
Gather-Scatter DRAM In-DRAM Address Translation to Improve the Spatial Locality of Non-unit Strided Accesses
Gather-Scatter DRAM In-DRAM Address Translation to Improve the Spatial Locality of Non-unit Strided Accesses Vivek Seshadri Thomas Mullins, AmiraliBoroumand, Onur Mutlu, Phillip B. Gibbons, Michael A.
Spatial Memory Streaming (with rotated patterns)
Spatial Memory Streaming (with rotated patterns) Michael Ferdman, Stephen Somogyi, and Babak Falsafi Computer Architecture Lab at 2006 Stephen Somogyi The Memory Wall Memory latency 100 s clock cycles;
A Comparison of Capacity Management Schemes for Shared CMP Caches
A Comparison of Capacity Management Schemes for Shared CMP Caches Carole-Jean Wu and Margaret Martonosi Princeton University 7 th Annual WDDD 6/22/28 Motivation P P1 P1 Pn L1 L1 L1 L1 Last Level On-Chip
SWAP: EFFECTIVE FINE-GRAIN MANAGEMENT
: EFFECTIVE FINE-GRAIN MANAGEMENT OF SHARED LAST-LEVEL CACHES WITH MINIMUM HARDWARE SUPPORT Xiaodong Wang, Shuang Chen, Jeff Setter, and José F. Martínez Computer Systems Lab Cornell University Page 1
Comparing Memory Systems for Chip Multiprocessors
Comparing Memory Systems for Chip Multiprocessors Jacob Leverich Hideho Arakida, Alex Solomatnikov, Amin Firoozshahian, Mark Horowitz, Christos Kozyrakis Computer Systems Laboratory Stanford University
Data Criticality in Network-On-Chip Design. Joshua San Miguel Natalie Enright Jerger
Data Criticality in Network-On-Chip Design Joshua San Miguel Natalie Enright Jerger Network-On-Chip Efficiency Efficiency is the ability to produce results with the least amount of waste. Wasted time Wasted
Micro-sector Cache: Improving Space Utilization in Sectored DRAM Caches
Micro-sector Cache: Improving Space Utilization in Sectored DRAM Caches Mainak Chaudhuri Mukesh Agrawal Jayesh Gaur Sreenivas Subramoney Indian Institute of Technology, Kanpur 286, INDIA Intel Architecture
TDT Coarse-Grained Multithreading. Review on ILP. Multi-threaded execution. Contents. Fine-Grained Multithreading
Review on ILP TDT 4260 Chap 5 TLP & Hierarchy What is ILP? Let the compiler find the ILP Advantages? Disadvantages? Let the HW find the ILP Advantages? Disadvantages? Contents Multi-threading Chap 3.5
Staged Memory Scheduling
Staged Memory Scheduling Rachata Ausavarungnirun, Kevin Chang, Lavanya Subramanian, Gabriel H. Loh*, Onur Mutlu Carnegie Mellon University, *AMD Research June 12 th 2012 Executive Summary Observation:
Thread Cluster Memory Scheduling: Exploiting Differences in Memory Access Behavior. Yoongu Kim Michael Papamichael Onur Mutlu Mor Harchol-Balter
Thread Cluster Memory Scheduling: Exploiting Differences in Memory Access Behavior Yoongu Kim Michael Papamichael Onur Mutlu Mor Harchol-Balter Motivation Memory is a shared resource Core Core Core Core
Computing on the Lunatic Fringe: Exascale Computers and Why You Should Care. Mattan Erez The University of Texas at Austin
1 Computing on the Lunatic Fringe: Exascale Computers and Why You Should Care Mattan Erez The University of Texas at Austin (C) Mattan Erez 2 Arch-focused whole-system approach Efficiency requirements
Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation!
Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation! Xiangyao Yu 1, Christopher Hughes 2, Nadathur Satish 2, Onur Mutlu 3, Srinivas Devadas 1 1 MIT 2 Intel Labs 3 ETH Zürich 1 High-Bandwidth
OpenPrefetch. (in-progress)
OpenPrefetch Let There Be Industry-Competitive Prefetching in RISC-V Processors (in-progress) Bowen Huang, Zihao Yu, Zhigang Liu, Chuanqi Zhang, Sa Wang, Yungang Bao Institute of Computing Technology(ICT),
Fall 2011 Prof. Hyesoon Kim. Thanks to Prof. Loh & Prof. Prvulovic
Fall 2011 Prof. Hyesoon Kim Thanks to Prof. Loh & Prof. Prvulovic Reading: Data prefetch mechanisms, Steven P. Vanderwiel, David J. Lilja, ACM Computing Surveys, Vol. 32, Issue 2 (June 2000) If memory
Rethinking On-chip DRAM Cache for Simultaneous Performance and Energy Optimization
Rethinking On-chip DRAM Cache for Simultaneous Performance and Energy Optimization Fazal Hameed and Jeronimo Castrillon Center for Advancing Electronics Dresden (cfaed), Technische Universität Dresden,
MULTIPROCESSORS AND THREAD-LEVEL. B649 Parallel Architectures and Programming
MULTIPROCESSORS AND THREAD-LEVEL PARALLELISM B649 Parallel Architectures and Programming Motivation behind Multiprocessors Limitations of ILP (as already discussed) Growing interest in servers and server-performance
A Case for Core-Assisted Bottleneck Acceleration in GPUs Enabling Flexible Data Compression with Assist Warps
A Case for Core-Assisted Bottleneck Acceleration in GPUs Enabling Flexible Data Compression with Assist Warps Nandita Vijaykumar Gennady Pekhimenko, Adwait Jog, Abhishek Bhowmick, Rachata Ausavarangnirun,
Shared Memory Multiprocessors. Symmetric Shared Memory Architecture (SMP) Cache Coherence. Cache Coherence Mechanism. Interconnection Network
Shared Memory Multis Processor Processor Processor i Processor n Symmetric Shared Memory Architecture (SMP) cache cache cache cache Interconnection Network Main Memory I/O System Cache Coherence Cache
MULTIPROCESSORS AND THREAD-LEVEL PARALLELISM. B649 Parallel Architectures and Programming
MULTIPROCESSORS AND THREAD-LEVEL PARALLELISM B649 Parallel Architectures and Programming Motivation behind Multiprocessors Limitations of ILP (as already discussed) Growing interest in servers and server-performance
arxiv: v1 [cs.ar] 10 Apr 2017
![arxiv: v1 [cs.ar] 10 Apr 2017](/thumbs/87/95719120.jpg "arxiv: v1 [cs.ar] 10 Apr 2017")
Banshee: Bandwidth-Efficient DRAM Caching Via Software/Hardware Cooperation Xiangyao Yu, Christopher J. Hughes, Nadathur Satish, Onur Mutlu, Srinivas Devadas CSAIL, MIT, Intel Labs, ETH Zurich {yxy, devadas}@mit.edu,
Lecture 5: Refresh, Chipkill. Topics: refresh basics and innovations, error correction
Lecture 5: Refresh, Chipkill Topics: refresh basics and innovations, error correction 1 Refresh Basics A cell is expected to have a retention time of 64ms; every cell must be refreshed within a 64ms window
Evaluating STT-RAM as an Energy-Efficient Main Memory Alternative
Evaluating STT-RAM as an Energy-Efficient Main Memory Alternative Emre Kültürsay *, Mahmut Kandemir *, Anand Sivasubramaniam *, and Onur Mutlu * Pennsylvania State University Carnegie Mellon University
Decoupled Compressed Cache: Exploiting Spatial Locality for Energy-Optimized Compressed Caching
Decoupled Compressed Cache: Exploiting Spatial Locality for Energy-Optimized Compressed Caching Somayeh Sardashti and David A. Wood University of Wisconsin-Madison 1 Please find the power point presentation
A Locality-Aware Memory Hierarchy for Energy-Efficient GPU Architectures
A Locality-Aware Memory Hierarchy for Energy-Efficient GPU Architectures Minsoo Rhu Michael Sullivan Jingwen Leng Mattan Erez {minsoo.rhu, mbsullivan, jingwen, mattan.erez}@utexas.edu Department of Electrical
Emerging NVM Memory Technologies
Emerging NVM Memory Technologies Yuan Xie Associate Professor The Pennsylvania State University Department of Computer Science & Engineering www.cse.psu.edu/~yuanxie yuanxie@cse.psu.edu Position Statement
COSC 6385 Computer Architecture - Memory Hierarchies (II)
COSC 6385 Computer Architecture - Memory Hierarchies (II) Edgar Gabriel Spring 2018 Types of cache misses Compulsory Misses: first access to a block cannot be in the cache (cold start misses) Capacity
HRL: Efficient and Flexible Reconfigurable Logic for Near-Data Processing
HRL: Efficient and Flexible Reconfigurable Logic for Near-Data Processing Mingyu Gao and Christos Kozyrakis Stanford University http://mast.stanford.edu HPCA March 14, 2016 PIM is Coming Back End of Dennard
LECTURE 5: MEMORY HIERARCHY DESIGN
LECTURE 5: MEMORY HIERARCHY DESIGN Abridged version of Hennessy & Patterson (2012):Ch.2 Introduction Programmers want unlimited amounts of memory with low latency Fast memory technology is more expensive
High Performance Memory Access Scheduling Using Compute-Phase Prediction and Writeback-Refresh Overlap
High Performance Memory Access Scheduling Using Compute-Phase Prediction and Writeback-Refresh Overlap Yasuo Ishii Kouhei Hosokawa Mary Inaba Kei Hiraki The University of Tokyo, 7-3-1, Hongo Bunkyo-ku,
Maximizing Server Efficiency from μarch to ML accelerators. Michael Ferdman
Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency from μarch to ML accelerators Michael Ferdman Maximizing Server Efficiency with ML accelerators Michael
Computer Architecture A Quantitative Approach, Fifth Edition. Chapter 2. Memory Hierarchy Design. Copyright 2012, Elsevier Inc. All rights reserved.
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 2 Memory Hierarchy Design 1 Introduction Programmers want unlimited amounts of memory with low latency Fast memory technology is more
ECE 485/585 Midterm Exam
ECE 485/585 Midterm Exam Time allowed: 100 minutes Total Points: 65 Points Scored: Name: Problem No. 1 (12 points) For each of the following statements, indicate whether the statement is TRUE or FALSE:
Linearly Compressed Pages: A Main Memory Compression Framework with Low Complexity and Low Latency
Linearly Compressed Pages: A Main Memory Compression Framework with Low Complexity and Low Latency Gennady Pekhimenko, Vivek Seshadri, Yoongu Kim, Hongyi Xin, Onur Mutlu, Todd C. Mowry Phillip B. Gibbons,
Copyright 2012, Elsevier Inc. All rights reserved.
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 2 Memory Hierarchy Design 1 Introduction Introduction Programmers want unlimited amounts of memory with low latency Fast memory technology
Computer Architecture. A Quantitative Approach, Fifth Edition. Chapter 2. Memory Hierarchy Design. Copyright 2012, Elsevier Inc. All rights reserved.
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 2 Memory Hierarchy Design 1 Programmers want unlimited amounts of memory with low latency Fast memory technology is more expensive per
Designing High-Performance and Fair Shared Multi-Core Memory Systems: Two Approaches. Onur Mutlu March 23, 2010 GSRC
Designing High-Performance and Fair Shared Multi-Core Memory Systems: Two Approaches Onur Mutlu onur@cmu.edu March 23, 2010 GSRC Modern Memory Systems (Multi-Core) 2 The Memory System The memory system
Lecture 11 Cache. Peng Liu.
Lecture 11 Cache Peng Liu liupeng@zju.edu.cn 1 Associative Cache Example 2 Associative Cache Example 3 Associativity Example Compare 4-block caches Direct mapped, 2-way set associative, fully associative
EECS 470. Lecture 14 Advanced Caches. DEC Alpha. Fall Jon Beaumont
Lecture 14 Advanced Caches DEC Alpha Fall 2018 Instruction Cache BIU Jon Beaumont www.eecs.umich.edu/courses/eecs470/ Data Cache Slides developed in part by Profs. Austin, Brehob, Falsafi, Hill, Hoe, Lipasti,
Lecture 7 - Memory Hierarchy-II
CS 152 Computer Architecture and Engineering Lecture 7 - Memory Hierarchy-II John Wawrzynek Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~johnw
Copyright 2012, Elsevier Inc. All rights reserved.
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 2 Memory Hierarchy Design 1 Introduction Programmers want unlimited amounts of memory with low latency Fast memory technology is more
Why memory hierarchy? Memory hierarchy. Memory hierarchy goals. CS2410: Computer Architecture. L1 cache design. Sangyeun Cho
Why memory hierarchy? L1 cache design Sangyeun Cho Computer Science Department Memory hierarchy Memory hierarchy goals Smaller Faster More expensive per byte CPU Regs L1 cache L2 cache SRAM SRAM To provide
Performance Characterization, Prediction, and Optimization for Heterogeneous Systems with Multi-Level Memory Interference
The 2017 IEEE International Symposium on Workload Characterization Performance Characterization, Prediction, and Optimization for Heterogeneous Systems with Multi-Level Memory Interference Shin-Ying Lee
A Row Buffer Locality-Aware Caching Policy for Hybrid Memories. HanBin Yoon Justin Meza Rachata Ausavarungnirun Rachael Harding Onur Mutlu
A Row Buffer Locality-Aware Caching Policy for Hybrid Memories HanBin Yoon Justin Meza Rachata Ausavarungnirun Rachael Harding Onur Mutlu Overview Emerging memories such as PCM offer higher density than
Portland State University ECE 587/687. Caches and Memory-Level Parallelism
Portland State University ECE 587/687 Caches and Memory-Level Parallelism Revisiting Processor Performance Program Execution Time = (CPU clock cycles + Memory stall cycles) x clock cycle time For each
AB-Aware: Application Behavior Aware Management of Shared Last Level Caches
AB-Aware: Application Behavior Aware Management of Shared Last Level Caches Suhit Pai, Newton Singh and Virendra Singh Computer Architecture and Dependable Systems Laboratory Department of Electrical Engineering
CS425 Computer Systems Architecture
CS425 Computer Systems Architecture Fall 2017 Thread Level Parallelism (TLP) CS425 - Vassilis Papaefstathiou 1 Multiple Issue CPI = CPI IDEAL + Stalls STRUC + Stalls RAW + Stalls WAR + Stalls WAW + Stalls
ECE 669 Parallel Computer Architecture
ECE 669 Parallel Computer Architecture Lecture 9 Workload Evaluation Outline Evaluation of applications is important Simulation of sample data sets provides important information Working sets indicate
Execution-based Prediction Using Speculative Slices
Execution-based Prediction Using Speculative Slices Craig Zilles and Guri Sohi University of Wisconsin - Madison International Symposium on Computer Architecture July, 2001 The Problem Two major barriers
The Memory Hierarchy. Cache, Main Memory, and Virtual Memory (Part 2)
The Memory Hierarchy Cache, Main Memory, and Virtual Memory (Part 2) Lecture for CPSC 5155 Edward Bosworth, Ph.D. Computer Science Department Columbus State University Cache Line Replacement The cache
Lecture 14: Memory Hierarchy. James C. Hoe Department of ECE Carnegie Mellon University
18 447 Lecture 14: Memory Hierarchy James C. Hoe Department of ECE Carnegie Mellon University 18 447 S18 L14 S1, James C. Hoe, CMU/ECE/CALCM, 2018 Your goal today Housekeeping understand memory system
Lecture 25: Busses. A Typical Computer Organization
S 09 L25-1 18-447 Lecture 25: Busses James C. Hoe Dept of ECE, CMU April 27, 2009 Announcements: Project 4 due this week (no late check off) HW 4 due today Handouts: Practice Final Solutions A Typical
Announcements. ECE4750/CS4420 Computer Architecture L6: Advanced Memory Hierarchy. Edward Suh Computer Systems Laboratory
ECE4750/CS4420 Computer Architecture L6: Advanced Memory Hierarchy Edward Suh Computer Systems Laboratory suh@csl.cornell.edu Announcements Lab 1 due today Reading: Chapter 5.1 5.3 2 1 Overview How to
4.1 Introduction 4.3 Datapath 4.4 Control 4.5 Pipeline overview 4.6 Pipeline control * 4.7 Data hazard & forwarding * 4.
Chapter 4: CPU 4.1 Introduction 4.3 Datapath 4.4 Control 4.5 Pipeline overview 4.6 Pipeline control * 4.7 Data hazard & forwarding * 4.8 Control hazard 4.14 Concluding Rem marks Hazards Situations that
Advanced Computer Architecture
ECE 563 Advanced Computer Architecture Fall 2009 Lecture 3: Memory Hierarchy Review: Caches 563 L03.1 Fall 2010 Since 1980, CPU has outpaced DRAM... Four-issue 2GHz superscalar accessing 100ns DRAM could
Minimalist Open-page: A DRAM Page-mode Scheduling Policy for the Many-core Era
Minimalist Open-page: A DRAM Page-mode Scheduling Policy for the Many-core Era Dimitris Kaseridis Electrical and Computer Engineering The University of Texas at Austin Austin, TX, USA kaseridis@mail.utexas.edu
Introduction to Parallel Programming Models
Introduction to Parallel Programming Models Tim Foley Stanford University Beyond Programmable Shading 1 Overview Introduce three kinds of parallelism Used in visual computing Targeting throughput architectures
Simultaneous Multithreading on Pentium 4
Hyper-Threading: Simultaneous Multithreading on Pentium 4 Presented by: Thomas Repantis trep@cs.ucr.edu CS203B-Advanced Computer Architecture, Spring 2004 p.1/32 Overview Multiple threads executing on
COSC 6385 Computer Architecture - Memory Hierarchy Design (III)
COSC 6385 Computer Architecture - Memory Hierarchy Design (III) Fall 2006 Reducing cache miss penalty Five techniques Multilevel caches Critical word first and early restart Giving priority to read misses
Cache Performance and Memory Management: From Absolute Addresses to Demand Paging. Cache Performance
6.823, L11--1 Cache Performance and Memory Management: From Absolute Addresses to Demand Paging Asanovic Laboratory for Computer Science M.I.T. http://www.csg.lcs.mit.edu/6.823 Cache Performance 6.823,
The University of Texas at Austin
EE382N: Principles in Computer Architecture Parallelism and Locality Fall 2009 Lecture 24 Stream Processors Wrapup + Sony (/Toshiba/IBM) Cell Broadband Engine Mattan Erez The University of Texas at Austin
Computer Systems Architecture
Computer Systems Architecture Lecture 24 Mahadevan Gomathisankaran April 29, 2010 04/29/2010 Lecture 24 CSCE 4610/5610 1 Reminder ABET Feedback: http://www.cse.unt.edu/exitsurvey.cgi?csce+4610+001 Student
Chapter 5 Large and Fast: Exploiting Memory Hierarchy (Part 1)
Department of Electr rical Eng ineering, Chapter 5 Large and Fast: Exploiting Memory Hierarchy (Part 1) 王振傑 (Chen-Chieh Wang) ccwang@mail.ee.ncku.edu.tw ncku edu Depar rtment of Electr rical Engineering,
15-740/ Computer Architecture Lecture 20: Main Memory II. Prof. Onur Mutlu Carnegie Mellon University
15-740/18-740 Computer Architecture Lecture 20: Main Memory II Prof. Onur Mutlu Carnegie Mellon University Today SRAM vs. DRAM Interleaving/Banking DRAM Microarchitecture Memory controller Memory buses
Lecture 15: DRAM Main Memory Systems. Today: DRAM basics and innovations (Section 2.3)
Lecture 15: DRAM Main Memory Systems Today: DRAM basics and innovations (Section 2.3) 1 Memory Architecture Processor Memory Controller Address/Cmd Bank Row Buffer DIMM Data DIMM: a PCB with DRAM chips
Computer Architecture Spring 2016
omputer Architecture Spring 2016 Lecture 09: Prefetching Shuai Wang Department of omputer Science and Technology Nanjing University Prefetching(1/3) Fetch block ahead of demand Target compulsory, capacity,
Microarchitecture Overview. Performance
Microarchitecture Overview Prof. Scott Rixner Duncan Hall 3028 rixner@rice.edu January 15, 2007 Performance 4 Make operations faster Process improvements Circuit improvements Use more transistors to make
Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation
Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation Xiangyao Yu 1 Christopher J. Hughes 2 Nadathur Satish 2 Onur Mutlu 3 Srinivas Devadas 1 1 MIT 2 Intel Labs 3 ETH Zürich ABSTRACT
Lecture 17: Memory Hierarchy: Cache Design
S 09 L17-1 18-447 Lecture 17: Memory Hierarchy: Cache Design James C. Hoe Dept of ECE, CMU March 24, 2009 Announcements: Project 3 is due Midterm 2 is coming Handouts: Practice Midterm 2 solutions The
CS 152 Computer Architecture and Engineering. Lecture 7 - Memory Hierarchy-II
CS 152 Computer Architecture and Engineering Lecture 7 - Memory Hierarchy-II Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~krste!
Multilevel Memories. Joel Emer Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology
1 Multilevel Memories Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology Based on the material prepared by Krste Asanovic and Arvind CPU-Memory Bottleneck 6.823
EE382N (20): Computer Architecture - Parallelism and Locality Fall 2011 Lecture 23 Memory Systems
EE382 (20): Computer Architecture - Parallelism and Locality Fall 2011 Lecture 23 Memory Systems Mattan Erez The University of Texas at Austin EE382: Principles of Computer Architecture, Fall 2011 -- Lecture
Towards Bandwidth-Efficient Prefetching with Slim AMPM
Towards Bandwidth-Efficient Prefetching with Slim Vinson Young School of Electrical and Computer Engineering Georgia Institute of Technology Atlanta, Georgia 30332 0250 Email: vyoung@gatech.edu Ajit Krisshna
SOFTWARE-DEFINED MEMORY HIERARCHIES: SCALABILITY AND QOS IN THOUSAND-CORE SYSTEMS
SOFTWARE-DEFINED MEMORY HIERARCHIES: SCALABILITY AND QOS IN THOUSAND-CORE SYSTEMS DANIEL SANCHEZ MIT CSAIL IAP MEETING MAY 21, 2013 Research Agenda Lack of technology progress Moore s Law still alive Power
Computer and Information Sciences College / Computer Science Department CS 207 D. Computer Architecture. Lecture 9: Multiprocessors
Computer and Information Sciences College / Computer Science Department CS 207 D Computer Architecture Lecture 9: Multiprocessors Challenges of Parallel Processing First challenge is % of program inherently
Introduction to OpenMP. Lecture 10: Caches
Introduction to OpenMP Lecture 10: Caches Overview Why caches are needed How caches work Cache design and performance. The memory speed gap Moore s Law: processors speed doubles every 18 months. True for
CS 152 Computer Architecture and Engineering. Lecture 7 - Memory Hierarchy-II
CS 152 Computer Architecture and Engineering Lecture 7 - Memory Hierarchy-II Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~krste
Data Storage and Query Answering. Data Storage and Disk Structure (2)
Data Storage and Query Answering Data Storage and Disk Structure (2) Review: The Memory Hierarchy Swapping, Main-memory DBMS s Tertiary Storage: Tape, Network Backup 3,200 MB/s (DDR-SDRAM @200MHz) 6,400
EECS750: Advanced Operating Systems. 2/24/2014 Heechul Yun
EECS750: Advanced Operating Systems 2/24/2014 Heechul Yun 1 Administrative Project Feedback of your proposal will be sent by Wednesday Midterm report due on Apr. 2 3 pages: include intro, related work,
Caches and Memory Hierarchy: Review. UCSB CS240A, Winter 2016
Caches and Memory Hierarchy: Review UCSB CS240A, Winter 2016 1 Motivation Most applications in a single processor runs at only 10-20% of the processor peak Most of the single processor performance loss
Agenda. System Performance Scaling of IBM POWER6 TM Based Servers
System Performance Scaling of IBM POWER6 TM Based Servers Jeff Stuecheli Hot Chips 19 August 2007 Agenda Historical background POWER6 TM chip components Interconnect topology Cache Coherence strategies
Practical Near-Data Processing for In-Memory Analytics Frameworks
Practical Near-Data Processing for In-Memory Analytics Frameworks Mingyu Gao, Grant Ayers, Christos Kozyrakis Stanford University http://mast.stanford.edu PACT Oct 19, 2015 Motivating Trends End of Dennard
Improving Cache Performance using Victim Tag Stores
Improving Cache Performance using Victim Tag Stores SAFARI Technical Report No. 2011-009 Vivek Seshadri, Onur Mutlu, Todd Mowry, Michael A Kozuch {vseshadr,tcm}@cs.cmu.edu, onur@cmu.edu, michael.a.kozuch@intel.com
Introduction to parallel computers and parallel programming. Introduction to parallel computersand parallel programming p. 1
Introduction to parallel computers and parallel programming Introduction to parallel computersand parallel programming p. 1 Content A quick overview of morden parallel hardware Parallelism within a chip
Computer Architecture and System Software Lecture 09: Memory Hierarchy. Instructor: Rob Bergen Applied Computer Science University of Winnipeg
Computer Architecture and System Software Lecture 09: Memory Hierarchy Instructor: Rob Bergen Applied Computer Science University of Winnipeg Announcements Midterm returned + solutions in class today SSD
Cache Memory COE 403. Computer Architecture Prof. Muhamed Mudawar. Computer Engineering Department King Fahd University of Petroleum and Minerals
Cache Memory COE 403 Computer Architecture Prof. Muhamed Mudawar Computer Engineering Department King Fahd University of Petroleum and Minerals Presentation Outline The Need for Cache Memory The Basics
Lecture 20: Virtual Memory, Protection and Paging. Multi-Level Caches
S 09 L20-1 18-447 Lecture 20: Virtual Memory, Protection and Paging James C. Hoe Dept of ECE, CMU April 8, 2009 Announcements: Best class ever, next Monday Handouts: H14 HW#4 (on Blackboard), due 4/22/09
Lecture 24: Memory, VM, Multiproc
Lecture 24: Memory, VM, Multiproc Today s topics: Security wrap-up Off-chip Memory Virtual memory Multiprocessors, cache coherence 1 Spectre: Variant 1 x is controlled by attacker Thanks to bpred, x can
18-447: Computer Architecture Lecture 23: Tolerating Memory Latency II. Prof. Onur Mutlu Carnegie Mellon University Spring 2012, 4/18/2012
18-447: Computer Architecture Lecture 23: Tolerating Memory Latency II Prof. Onur Mutlu Carnegie Mellon University Spring 2012, 4/18/2012 Reminder: Lab Assignments Lab Assignment 6 Implementing a more