Memory. Memory Technologies

Size: px

Start display at page:

Download "Memory. Memory Technologies"

Everett May
5 years ago
Views:

1 Memory Memory technologies Memory hierarchy Cache basics Cache variations Virtual memory Synchronization Galen Sasaki EE 36 University of Hawaii Memory Technologies Read Only Memory (ROM) Static RAM (SRAM) Basic cell: clocked D latch (vs. D flip flop) Array of cells: -dimension and 2-dimen RAM access: timing Dynamic RAM (DRAM) Basic cell Faster DRAM, e.g. synchronous DRAM Galen Sasaki EE 36 University of Hawaii 2

2 ROM Random access means that the access time (to read or write) is not dependent on the address -- different from hard disks ROM is random access but read only Programmable ROM (PROM) Erasable PROM (EPROM) Electrically EPROM (EEPROM) Galen Sasaki EE 36 University of Hawaii 3 RAM RAM is read and writeable random access memory Memory is erased when power goes off ROM doesn t lose memory when power goes off Static RAM: faster but more expensive Dynamic RAM: slower, smaller, and less expensive but it s getting faster Galen Sasaki EE 36 University of Hawaii 4 2

3 Components: CMOS n-channel transistor High(H) gate source gate = --> close gate = 0 --> open A B A B drain Low(L) or GND p-channel transistor High(H) High(H) gate source drain gate = --> open gate = 0 --> close A=L closed open Galen Sasaki EE 36 University of Hawaii 5 B Low(L) A=H open closed Low(L) B C Switch and NOR High(H) A better for H A A C C C Switch: When C = then switch is closed, i.e., A = B. When C = 0, switch is open B better for L B B A=L B=L Low(L) High(H) closed closed open Low(L) Y Y at least one is open High(H) A or B = H open at least one is closed Low(L) Y closed Galen Sasaki EE 36 University of Hawaii 6 3

4 Simple Memory Cell: D Latch Clock D Q Clock Similar to a D flip flop D Clock Clock Clock Q Clock = L, It holds its value Clock = H, Q = D, i.e., it s transparent from input to output. Clock = H Clock = L D Hold Q D Q Transparent Galen Sasaki EE 36 University of Hawaii 7 D Flip Flop D Master D Q Slave D Q Q Clock Clock Clock D Clock = L D Q D Q Q Transparent =D Hold Hold Clock Clock H L D Clock = H D Q Hold Clock L last value D Q of D Transparent while Clock Clock H = L Galen Sasaki EE 36 University of Hawaii 8 Q 4

5 Common Type bus bus Address CS R/W Static RAM READ Address R/W CS Delay Chip Select (CS): Asserting it will turn on the RAM WRITE Address R/W: Asserting it, will allow you to read it Unasserting it, will allow you to write to it R/W CS Address should be stable while CS = Min Time Galen Sasaki EE 36 University of Hawaii 9 -Bit SRAM Cell RAM bus Address bit Let s build it in steps. First consider just R/W CS R/W D Q Clock R/W A -bit memory cell Add in CS D Q Clock CS CS R/W R/W Galen Sasaki EE 36 University of Hawaii 0 5

6 -Bit SRAM RAM 0 bus Demultiplexer CS Address CS 0 R/W bit CS R/W CS R/W a3 a2 row addr row CS R/W 5 Address CS R/W column a3 a2 Galen a a0 Sasaki EE 36 University of Hawaii R/W 2 3 a a0 column addr CS 32-Bit SRAM RAM bus Address CS R/W CS R/W Address Address Address 6 bit Address CS Address CS Address CS CSR/W Address Address CSR/W Address CSR/W Address CSR/W Address CSR/W Address CSR/W d3 CSR/W CSR/W R/W R/W d0 Galen Sasaki EE 36 University of Hawaii 2 6

7 Dynamic RAM Memory cells can forget! Cells must be refreshed Less number of devices per bit Read destroys bits Write bit back after a read Word line Word line Word line Word line Word line Word line cell cell cell cell cell cell A cell Word line Passive transistor Capacitor Bit line R/W CS Sense/write amplifiers -- sense and amplifier data data Galen Sasaki EE 36 University of Hawaii 3 4M x DRAM Row decoder -to x 2048 array Address[0-0] Column latches Mux Latch 2048 bits at a time Throw away 2047 Dout Galen Sasaki EE 36 University of Hawaii 4 7

8 Memory Hierarchy Motivation, i.e., Why? Cache basics Contents Reading Replacement policy Writing consistency Example: Direct Mapped Blocks of memory: better efficiency Galen Sasaki EE 36 University of Hawaii 5 Memory Hierarchy Making large fast memory cheaply. Memory Types Access Times Cost/Mbyte (93) Static RAM (SRAM) fast (8-35 ns) $ Dynamic RAM (DRAM) slower (90-20 ns) $25-50 Hard Disk even slower (0-20 M ns) $-2 CPU cache (fast) main memory (slower) Galen Sasaki EE 36 University of Hawaii 6 8

9 Cache: Why does it work? CPU cache (fast) main memory (slower) Principle of locality: observation about executing programs Temoporal locality (locality in time): if an item is referenced, it will tend to be referenced soon, e.g., loops Spatial locality (locality in space): if an item is referenced, items that have nearly the same address values will tend to be referenced soon. Galen Sasaki EE 36 University of Hawaii 7 Temporal Locality main: add $,$2,$3 andi $2,$5,9... loop: beq skip... j loop skip:.... spatial locality temporal locality Galen Sasaki EE 36 University of Hawaii 8 9

10 index cache 038 Cache holds copies of the contents of memory Memory access: read/write hit: find element in cache miss: element not in cache Cache Basics Replacement policy: if we store something new in the cache, where do we write it? Example: Least Recently Used (LRU) -- the oldest one. 038 Main Galen Sasaki EE 36 University of Hawaii 9 Cache Contents valid bit: used or unused tag main memory address-- it s an identifier of where the data comes from data contents from memory age used by LRU policy this is updated every time the memory address is accessed Galen Sasaki EE 36 University of Hawaii 20 0

11 Read Processor Step cache request addr 024 check if 024 is in cache (check tags) HIT main memory MISS Step 2 Step 2 Processor data for 024 cache update age of 024 main memory Processor cache req 024 main memory data for 024 load 024 in cache and (possibly) replace something. update age of 024 Galen Sasaki EE 36 University of Hawaii 2 Write Processor write addr 024 HIT Step cache check if 024 is in cache (check tags) main memory MISS Step 2 Step 2 Processor write addr 024 cache update age of 024 main memory? Depends on write policy Processor write addr 024 cache replace something main memory Galen Sasaki EE 36 University of Hawaii 22

12 Memory Write Three approaches:. Write-through: write to both cache and main memory. Simple, but large overhead. processor write fast cache slow memory 2. Write-buffer: write-through, but there is a buffer that takes care of the writing to memory. Buffer allows CPU to go onto the next operation If buffer is not finished before the next memory access then CPU stalls until the writethrough is finished. processor one or more writes buffer stalls occur if writes aren t cache completed before next instruction that needs the word memory Galen Sasaki EE 36 University of Hawaii Write-back: write to cache Memory Write Update main memory when item in cache is replaced. More complicated. write cache update only if necessary memory cache contents v tag data dirty bit : indicates if the contents have to be written back to memory if replaced in cache Galen Sasaki EE 36 University of Hawaii 24 2

13 Example: Direct Mapped Caching: Variations. Where do you place a new entry? Or which do you replace? E.g., Least Recently Used 2. In case of write, when do you write back into main memory? E.g., write-through (always) Direct Mapped deals with. You want to place the contents of memory location A into the cache. You use the value of A to determine this. Thus, A is always placed into the same position in the cache. (Different than LRU) Most common way is to use the last few bits of A. Easy to find data in cache (don t have to search entire contents) Galen Sasaki EE 36 University of Hawaii 25 Direct Mapped Example cache index cache Use the last two bits to determine the cache index to map memory contents to. Why the last two bits, rather than the first two bits? Main Galen Sasaki EE 36 University of Hawaii 26 3

14 Direct Mapped Example cache Main Window the size of the cache Galen Sasaki EE 36 University of Hawaii 27 Cache Contents valid bit: used unused data contents from memory tag main memory address minus last bits main memory address = tag + cache index (e.g., 0) Galen Sasaki EE 36 University of Hawaii 28 4

15 Direct Mapped Hardware address byte offset (2 bits) CPU index Cache Hit & v tag data = Galen Sasaki EE 36 University of Hawaii 29 Blocks of Memory * Speed things up by organizing memory into blocks of words. o Access blocks instead of words o Take advantage of spatial locality of references Address Block Number or Block Address Bit offset (2 bits) Block Offset (indicates where the word is in the block) Block 00 Block 0 Main Block 0 Block Galen Sasaki EE 36 University of Hawaii 30 5

16 Memory Organization Main memory Interleaved word block address Cache * Access time of block is approx. access time of a single word. block # block offset bit offset CPU Galen Sasaki EE 36 University of Hawaii 3 Cache address tag block # index block offset bit offset v tag CPU = Mux Hit & Galen Sasaki EE 36 University of Hawaii 32 6

17 Memory Hierarchy Miss rate 40% 35% 30% 25% 20% 5% 0% 5% Bigger blocks mean smaller miss rates BUT Higher miss penalties TRADE OFF 0% Block size (bytes) KB 8 KB 6 KB 64 KB 256 KB Galen Sasaki EE 36 University of Hawaii 33 Designing memory systems to support caches CPU CPU CPU bus & mux Cache Cache Cache bus wide bus Memory Memory one-word-wide wide memory interleaved Example: clock cycle to send address 5 clock cycles for each DRAM access initiated clock cycle to send a word of data Galen Sasaki EE 36 University of Hawaii 34 7

18 Memory Access READ: same as before Step. Check if word is in cache Step 2. HIT: Read word from cache MISS: Read block (and word) from main memory. Put block in cache WRITE: different for write through Step. Read block from main memory into cache Write into main memory and cache Galen Sasaki EE 36 University of Hawaii 35 Cache Variations Associative memory Set associate memory Two levels of caching Galen Sasaki EE 36 University of Hawaii 36 8

19 Associative Memory Cache Contents Each element contains: tag, data, valid bit, age Access Cache Check cache by checking all elements (check tags of all elements to see if there s a hit) Replacement policy: can be more intelligent to reduce misses E.g., least recently used (LRU): need to update age field E.g., random Lowers miss rate but increases hardware complexity (and maybe speed) Galen Sasaki EE 36 University of Hawaii 37 Associative Memory Hardware byte offset (2 bits) v addr age data v addr age data v addr age data v addr age data addre ss = = = = CPU & & & & Hit Hit Hit Hit Hit OR 4-to- Multiplexer Galen Sasaki EE 36 University of Hawaii 38 9

20 Set Associative * Associative: Reduces the chance of a miss. Large complexity because all blocks must be searched * Direct-Mapped: Simpler complexity * Set Associative (Generalization of above): A block is mapped to a set just as in direct mapped A block may be mapped anywhere within its set just as in associative Blocks Sets set index is the set address Cache Words n-way set associative means n blocks in a set Galen Sasaki EE 36 University of Hawaii 39 Set Associative Main Memory block 0 A block is a super word Block #s are the addresses block # (b bits) block size = 2 k words k bits block block 2 Word address 00 2 m sets m bits set index block 3 block 4 block 5 set 0 set n-way set associative cache associative with n blocks block 6 block 7 direct map to the set using set index set 2 m- - Galen Sasaki EE 36 University of Hawaii 40 20

21 Cache Performance A. CPU time = ( CPU execution clock cycles + memory-stall clock cycles) x clock cycle time. memory-stall clock cycles = read-stall cycles + write-stall cycles B. Read-stall cycles = reads/program * read miss rate * read miss penalty C. Write-stall cycles = writes/program * write miss rate * write miss penalty + write buffer stalls usually negligible D. Simplifying case: Write and read penalties are about the same, and write buffer stalls are negligible. Memory-stall cycles = memory access/program * miss rate * miss penalty Galen Sasaki EE 36 University of Hawaii 4 5% Memory Hierarchy 2% 9 % Miss rate 6 % 3 % 0 % O n e-wa y Tw o-way Four-w ay Eight-w ay Associativity KB 6 KB 2 KB 32 K B 4 KB 64 K B 8 KB 28 KB Galen Sasaki EE 36 University of Hawaii 42 2

22 Improving Performance Set associative memory Choose set and block size so that hardware is fast (hit time) minimize miss rate Two Level Cache CPU single chip CPU Min hit time Level cache direct mapped cache Min miss rate Lowers miss penalty of L cache Level 2 cache SRAM set associative LRU Main memory Main memory Galen Sasaki EE 36 University of Hawaii 43 Really big memory Virtual Memory E.g., there are 32 bits in an address, addressing up to around 4 billion bytes. 64 bit addresses address much much more. Much bigger than main memory. Virtual memory is this really big memory but data is physically stored in all kinds of stuff Galen Sasaki EE 36 University of Hawaii 44 22

23 Virtual Memory Main memory Really big memory virtual addresses Page 0 Pages instead of blocks Page Page 2 Pages are reasonably big chunks of bytes Pages are stored in main memory (physical memory) or hard disk Page 3 Page 4 Page 5 Page 6 Page 7 Hard disk Etc How do you keep track of where the pages are? Galen Sasaki EE 36 University of Hawaii 45 Virtual Memory Processor page table register virtual addresses Page 0 Page Address Translation Main memory a cache for hard disk Page Table does address translation page #s location Page 2 Page 3 5 Page 4 7 Page 5 Page 6 Page 7 Hard disk Etc Galen Sasaki EE 36 University of Hawaii 46 23

24 Accessing Virtual Memory virtual address virtual page # page offset Translation lookaside buffer (TLB) (It s a cache for the page table) location of page Page table Use offset to get the exact word or byte within the page Galen Sasaki EE 36 University of Hawaii 47 Virtual Memory Disk accesses are slow memory management can be done in software page sizes are large (0s of Kbytes) to mitigate slow access times of disks miss penalties are very large so associative memory least recently used or even more complicated strategies write-back rather than write-through Galen Sasaki EE 36 University of Hawaii 48 24

25 Virtual Memory Programs do not have to be in contiguous portions of memory. They can be on pages located anywhere. Note that strategy for virtual memory is different than for caches because of the high miss penalties Galen Sasaki EE 36 University of Hawaii 49 Buses Galen Sasaki EE 36 University of Hawaii 50 25

26 Memory Hierarchy Galen Sasaki EE 36 University of Hawaii 5 Memory Hierarchy Virtual address Virtual page number Page offset Translation Physical page number Page offset Physical address Galen Sasaki EE 36 University of Hawaii 52 26

27 Memory Hierarchy Page table register V irtu a l a dd ress Virtual page number Page offset 20 2 V a lid Physical page number Page table If 0 th en p ag e is no t pre sen t in m em o ry Physical page number Page offset Physical address Galen Sasaki EE 36 University of Hawaii 53 Virtual page number Memory Hierarchy Valid Page table Physical page or disk address Physical memory Disk storage Galen Sasaki EE 36 University of Hawaii 54 27

28 Memory Hierarchy V irtu al p ag e n u m b er V alid Ta g T LB P h ysical p a ge ad d ress 0 P hysical m em ory P ag e tab le Ph ysical p ag e V alid or d isk ad d ress D isk storag e Galen Sasaki EE 36 University of Hawaii 55 Memory Hierarchy V irtual address V irtu a l p a g e n u m b e r P a g e o ffs e t V a lid D irt y T a g P h y s ic a l p a g e n u m b e r T L B T L B h it 20 P h y sica l p a g e nu m be r P a g e o ffs e t P h ys ica l a d d re s s P hysical address tag C a c h e in d e x B y te o ffs e t V a lid T a g D a ta C a ch e 3 2 C a ch e h it Galen Sasaki EE 36 University of Hawaii 56 D a ta 28

29 Memory Hierarchy Virtual address TLB access TLB miss excep tion No TLB hit? Yes Physical address No Write? Yes Try to read data from cach e No Write access bit on? Yes Cach e miss stall No Cache hit? Yes Write p rotection exception Write data into cach e, update th e tag, and put th e data and th e address in to the write buffer Deliver data to the CPU Galen Sasaki EE 36 University of Hawaii 57 4 % Memory Hierarchy 2 % 0 % Miss rate per type 8 % 6 % 4 % 2 % C a p a c ity 0 % C a c h e s iz e (K B ) O n e - w a y F o u r -w a y T w o - w a y E ig h t- w a y Galen Sasaki EE 36 University of Hawaii 58 29

30 Memory Hierarchy 00 Improvement factor Ye ar C PU (f ast) C P U (slo w ) D R AM D R AM C PU (slo w ) C PU (fast) D R AM cycle tim es slo w P (M H z) Galen Sasaki EE 36 University of Hawaii 59 H i t Memory Hierarchy A d d re s s (s h o w i n g b it p o s i tio n s ) T a g In d e x B y t e o f f s e t D a t a In d e x 0 2 V a l id T a g D a ta Galen Sasaki EE 36 University of Hawaii 60 30

31 Memory Hierarchy A d d re s s (s h o w in g b it p o sitio n s ) H it 6 4 B y te o ffs e t D a ta 6 bits 32 bits Valid Tag D ata 6 K e n trie s Galen Sasaki EE 36 University of Hawaii 6 Memory Hierarchy Address (showing bit positions) Hit Tag Byte offset Index Block offset 6 bits 28 bits V Tag 4K entries Mux 32 Galen Sasaki EE 36 University of Hawaii 62 3

32 Memory Hierarchy A d d r ess In d e x 0 2 V T a g V Tag V T ag V T ag to - m u ltip lex o r H it D ata Galen Sasaki EE 36 University of Hawaii 63 Memory Hierarchy Direct mapped Set associative Block # Set # Fully associative Tag 2 Tag 2 Tag 2 Search Search Search Galen Sasaki EE 36 University of Hawaii 64 32

33 Memory Hierarchy CPU CPU CPU Cache Multiplexor Cache Cache Bus Bus Bus Memory Memory bank 0 Memory bank Memory bank 2 Memory bank 3 Memory b. Wide memory organization c. Interleaved memory organization a. One-word-wide memory organization Galen Sasaki EE 36 University of Hawaii 65 Memory Hierarchy O n e - w a y se t a ss o cia tiv e (d ire c t m a p p e d ) B lo c k T a g D a ta S e t T w o - w a y se t a ss o cia tiv e T a g D a ta T a g D a ta S e t 0 F o u r- w a y s e t a s s o c ia tiv e T ag Tag Tag Tag E ig h t- w a y se t a s so c ia tiv e (fu lly a ss o c ia tiv e ) Tag D ata T ag D ata Tag Tag Tag Tag Tag Tag D ata Galen Sasaki EE 36 University of Hawaii 66 33

34 Memory Hierarchy CPU Levels in the memory hierarchy Level Level 2 Increasing distance from the CPU in access time Level n Size of the memory at each level Galen Sasaki EE 36 University of Hawaii 67 Memory Hierarchy Speed CPU Size Cost ($/bit) Fastest Memory Smallest Highest Memory Slowest Memory Biggest Lowest Galen Sasaki EE 36 University of Hawaii 68 34

35 Memory Hierarchy X4 X Xn 2 X4 X Xn 2 Xn X2 Xn X2 Xn X3 a. Before the reference to Xn X3 b. After the reference to Xn Galen Sasaki EE 36 University of Hawaii 69 Memory Hierarchy Cache Memory Galen Sasaki EE 36 University of Hawaii 70 35

36 Memory Hierarchy Processor are transferred Galen Sasaki EE 36 University of Hawaii 7 36

Chapter Seven. Large & Fast: Exploring Memory Hierarchy

Chapter Seven. Large & Fast: Exploring Memory Hierarchy Chapter Seven Large & Fast: Exploring Memory Hierarchy 1 Memories: Review SRAM (Static Random Access Memory): value is stored on a pair of inverting gates very fast but takes up more space than DRAM DRAM