Memory hierarchy Outline

Size: px

Start display at page:

Download "Memory hierarchy Outline"

Candice Todd
6 years ago
Views:

1 Memory hierarchy Outline Performance impact Principles of memory hierarchy Memory technology and basics 2 Page 1

2 Performance impact Memory references of a program typically determine the ultimate performance of a program Ideal pipelining assumes memory accesses can be done always in one cycle In practice, this is not always possible Example: Consider the following code sequence: lw r2,0x20 lw r3,0x30 add r1,r4,r5 sw r6,0x40 Assume: a) 1-cycle access to memory b) 4-cycle access to memory (for loads/stores, 1 for fetch) 3 Performance impact (2) a) 1-cycle memory access: 9 cycles lw r2,0x20 lw r3,0x30 add r1,r4,r5 sw r6,0x40 IF ID IF EX ID IF M EX ID IF WB M S S WB EX ID M EX WB M WB b) 4-cycles memory access: 17 cycles lw r2,0x20 lw r3,0x30 add r1,r4,r5 sw r6,0x40 IF ID IF EX ID IF M EX ID IF M M M WB S S S M M M S S S S S S S S S ID S S M EX S M EX WB M M M M WB 4 Page 2

3 Performance impact (3) Need to hide (external) memory latency Give the illusion of a fast, large & cheap memory system Solution: use a memory hierarchy 5 Principles of memory hierarchy 6 Page 3

4 Typical memory hierarchy cache virtual memory CPU CPU regs regs C 4 B a c 32 B 4 KB Memory h e disk disk register reference cache reference memory reference disk memory reference size: speed: $/Mbyte: block size: 200 B ~1 ns 4 B 32KB -- 4MB 5 ns ~1$/MB 16 B 256 MB 60 ns ~$0.5/MB 4 KB 10GB 10 ms $0.01/MB larger, slower, cheaper 7 Memories in a processor [Microprocessor Report 9/12/94] Caches: L1 data L1 instruction L2 unified TLB Branch history see page 8 of introduction. 8 Page 4

5 Memories in a processor (3) [Burger] 9 Locality of reference Why memory hierarchy works? Principle of Locality programs tend to reuse data and instructions near those they have used recently. Temporal locality: recently referenced items are likely to be referenced in the near future. Spatial locality: items with nearby addresses tend to be referenced close together in time. 10 Page 5

6 Sources of locality Data Reference array elements in succession (spatial) Instruction Reference instructions in sequence (spatial) Cycle through loop repeatedly (temporal) Example: sum = 0; for (i = 0; i < n; i++) sum += a[i]; *v = sum; Abstract Version of Machine Code I0: sum <-- 0 I1: ap <-- a I2: i <-- 0 I3: if (i >= n) goto done I4: loop: t <-- *ap I5: sum <-- sum + t I6: ap <-- ap + 4 I7: i <-- i + 1 I8: if (i < n) goto loop I9: done: *v <-- sum Memory Layout 0x0FC I0 0x100 I1 0x104 I2 0x108 I3 0x10C I4 0x110 I5 0x114 I6 0x400 a[0] 0x404 a[1] 0x408 a[2] 0x40C a[3] 0x410 a[4] 0x414 a[5] 0x7A4 *v 11 Exploting locality Temporal Locality Keep most recently accessed data items closer to the processor Spatial Locality Move data closer to processor in blocks (groups of contiguous words) Block = minimum unit of data between 2 levels Upper-level blocks are a subset of lower-level blocks 12 Page 6

7 Memory hierarchy: terminology Hit: required data appears in some block in the upper level Hit Rate: the fraction of memory accesses found in the upper level Hit Time: Time to access the upper level = access time + Time to determine hit/miss Miss: required data needs to be retrieved from a block in the lower level Miss Rate = 1 - Hit Rate Miss Penalty: Time to replace a block in the upper level = access time + transfer time To be effective: Hit Time << Miss Penalty 13 Accessing data in a memory hierarchy Between any two levels, memory divided into blocks Data moves between levels in block-sized chunks Upper-level blocks are a subset of lower-level blocks Example Access word w in block a (hit) w Access word v in block b (miss) v Upper a a a b b Lower a b a b a b 14 Page 7

8 Memory Basics & Technology Outline Basic memory technologies: SRAMs DRAMs Other types of memory 16 Page 8

9 Importance of technology Locality is only one side of the picture Memory hierarchy principle is limited by cost & technological issues Faster memories are expensive Slower memories memories are cheap Need to understand why 17 Static RAM (SRAM) Fast ~5 ns (depends on size) Persistent as long as power is supplied no refresh required Expensive O(1$)/MByte 6 transistors/bit Stable High immunity to noise and environmental disturbances Same process as standard logic (CMOS) Base technology for caches 18 Page 9

10 Anatomy of an SRAM bit (cell) Read: Set bit lines high (precharge) Set word line high See which bit line goes low Write: Set bit lines to opposite values Set word line Flip cell to new state bit line bit line b b (6 transistors) word line 19 Example 1-level-decode SRAM (16 x 8) W0 b7 b7 b1 b1 b0 b0 A0 W1 A1 A2 Address Address memory cells A3 W15 R/W Input/output lines d7 d1 d0 20 Page 10

11 Slower than SRAM Dynamic RAM (DRAM) Access time O(50ns) (depends on size) Concept of cycle time > access time Cycle time = min. time between consecutive accesses Non-persistent every row must be accessed every ~1 ms (refreshed) Cheaper than SRAM O(0.1$)/MByte 1 transistor/bit Fragile electrical noise, light, radiation Different process w.r.t. standard logic Requires extra fabrication steps Workhorse memory technology 21 Anatomy of a DRAM Cell Write: Drive bit line Select row Read: Precharge bit line to 1 Select row Cell and bit line share charges Very small voltage changes on the bit line Sense Restore the value Refresh Just do a dummy read to every cell. Bit Line C BL Access Transistor Word Line Storage Node C node 22 Page 11

12 Addressing arrays with bits Impossible to decode n address lines to 2 n row values Partition into bi-dimensional addressing Memory array seen as an R x C array of addresses R = 2 r, C = 2 c For each address: row(address) = address/c = leftmost r bits of address col(address) = address%c = righmost c bits of address address = r bits row c bits col row 1 col 2 23 Example 2-level decode DRAM (64Kx1) RAS 256 Rows row Row Row address address 8 \ Row Row 256x x256 cell cell array array A7-A0 col 256 Columns column column R/W Column Column address address 8 \ column column and and CAS Dout Din 24 Page 12

13 Row Address (~50ns) DRAM Operation Set Row address on address lines & strobe RAS Entire row read & stored in column es Contents of row of memory cells destroyed Column Address (~10ns) Set Column address on address lines & strobe CAS Access selected bit READ: transfer from selected column to Dout WRITE: Set selected column to Din Rewrite (~30ns) Write back entire row 25 Observations About DRAMs Timing Access time = 60ns < cycle time ~= 90ns Need to rewrite row Must refresh periodically Perform complete memory cycle for each row Approx. every 1ms sqrt(n) cycles Handled in background by memory controller Inefficient way to get single bit Effectively read entire row of sqrt(n) bits 26 Page 13

14 Enhanced Performance DRAMs Conventional Access Row + Col RAS CAS RAS CAS... Page Mode Row + Series of columns RAS CAS CAS CAS... Gives successive bits A7-A0 row col RAS Row Row address address Column Column address address 8 \ 8 \ Row Row 256x x256 cell array cell array R/W column column and and CAS Entire row buffered here Typical Performance row access time col access time cycle time page mode cycle time 50ns 10ns 90ns 25ns 27 SRAM SRAMs vs. DRAMs Technology quite stable fast, expensive, larger silicon footprint DRAM Several technological variants SDRAM, RAMBUS, DRAM optimized for capacity more than for speed slow, cheap, smaller silicon footprint Use SRAM in fast caches and DRAM in main memory 28 Page 14

15 SRAMs vs. DRAMs (2) SRAM metric /1980 $/MB 19,200 2, access (ns) DRAM metric /1980 $/MB 8, access (ns) typical size(mb) culled from back issues of Byte and PC Magazine 29 Storage price/mbyte $/Mbyte today SRAM DRAM disk Year [Byte and PC Magazine] 30 Page 15

16 Storage access times 100,000,000 10,000,000 disk DRAM 1,000,000 SRAM access time (ns) 100,000 10,000 1, today Year [Byte and PC Magazine] 31 Memory technology summary Cost and density improving at enormous rates. Speed lagging processor performance Memory hierarchies help narrow the gap: small fast SRAMS (cache) at upper levels large slow DRAMS (main memory) at lower levels Large & slow disks to back it all up Locality of reference makes it all work Keep most frequently accessed data in fastest memory 32 Page 16

Where Have We Been? Ch. 6 Memory Technology

Where Have We Been? Ch. 6 Memory Technology Where Have We Been? Combinational and Sequential Logic Finite State Machines Computer Architecture Instruction Set Architecture Tracing Instructions at the Register Level Building a CPU Pipelining Where