Large and Fast: Exploiting Memory Hierarchy

Transcription

1 CSE 431: Introduction to Operating Systems Large and Fast: Exploiting Memory Hierarchy Gojko Babić 10/5/018 Memory Hierarchy A computer system contains a hierarchy of storage devices with different costs, capacities, and access times. With a memory hierarchy, a faster storage device at one level of the hierarchy acts as a staging area for a slower storage device at the next lower level. Software that is well-written takes advantage of the hierarchy accessing the faster storage device at a particular level more frequently than the storage at the next level. As a programmer, understanding the memory hierarchy will result in better performance of applications. 1

2 An Example of Memory Hierarchy L0: Registers CPU registers hold words retrieved from L1 cache Smaller, faster, costlier per byte Larger, slower, cheaper per byte L4: L3: L: L1: L1 cache (SRAM) L cache (SRAM) Main memory (DRAM) Local secondary storage (local disks) L1 cache holds cache lines retrieved from L cache L cache holds cache lines retrieved from main memory Main memory holds disk blocks retrieved from local disks Local disks hold files retrieved from disks on remote network servers L5: Remote secondary storage (tapes, distributed file systems, Web servers) 3 The Levels in Memory Hierarchy Access time:< 0.5nsec nsec 50-70nsec 5-50μsec 5-0msec Technology: SRAM DRAM SSD magnetic disk Capacity: 10 3 bytes 10 6 bytes 10 9 bytes bytes 10 1 bytes Cost per GB: $500-$1,000 $10-$0 $0.75-$1.00 $0.05-$0.10 in 01 Apple Fusion Drive Higher the level, smaller and faster the memory. Try to keep most of the action in the higher levels. The principle of locality is the most important program property that is exploited in many parts of memory hierarchy. 4

3 Random Access Memory (RAM) Features Basic storage unit is a cell (one bit per cell); RAM is traditionally packaged as a chip; multiple chips form memory Static RAM (SRAM) Each cell implemented with a six-transistor circuit Relatively insensitive to disturbances such as electrical noise, radiation, etc. Faster and more expensive than DRAM Dynamic RAM (DRAM) Each bit stored as charge on a capacitor Value must be refreshed every ms Sensitive to disturbances Slower and cheaper than SRAM 5 SRAM vs DRAM Transistors Access Needs Sensitive? Cost Power Power Chip per bit time refresh? requirements dissipation density SRAM 4 or 6 1X No No 100x high high low DRAM 1 10X Yes Yes 1X low low high 6 3

4 Conventional DRAM Organization (d w) bit DRAM: d w organized as d supercells of size w 16 x 8 DRAM chip cols Memory controller (to/from CPU) / addr 8 / data rows supercell (,1) Internal row buffer 7 Reading 16x8 DRAM Supercell [,1] Step 1(a): Row access strobe (RAS) selects row. Step 1(b): Row copied from DRAM array to row buffer. 16 x 8 DRAM chip Memory controller RAS = / addr 0 1 Rows Cols / data 3 Internal row buffer 8 4

5 Reading 16x8 DRAM Supercell [,1] (cont.) Step (a): Column access strobe (CAS) selects column 1. Step (b): Supercell [,1] copied from buffer to data lines, and eventually back to the CPU. To CPU supercell [,1] Memory controller CAS = 1 / addr 8 / data 16 x 8 DRAM chip 0 1 Rows 3 Cols supercell [,1] Internal row buffer 9 Example: 4GB Memory Module addr (row = i, col = j) DRAM 7 DRAM 0 : supercell [i,j] 4 GB memory module consisting of eight 51Mx8 DRAMs bit doubleword at main memory address A Memory controller 64-bit doubleword 10 5

6 Enhanced DRAMs Enhanced DRAMS have optimizations that improve the speed with which the basic DRAM cells can be accessed. Examples: Fast page mode DRAM (FPM DRAM); up to 1996 Extended data out DRAM (EDO DRAM); Synchronous DRAM (SDRAM) Double Data-Rate Synchronous DRAM (DDR SDRAM) Rambus DRAM (RDRAM) 11 Growth of Capacity per DRAM chip 1 6

7 Prices of Six Generations of DRAMs 13 DRAM size increased by multiples of four approximately once every three years until 1996, and thereafter considerably slower. The improvements in access time have been slower but continuous, and cost roughly tracks density improvements, although cost is often affected by other issues, such as availability and demand. The cost per gibibyte (=10 9 bytes) is not adjusted for inflation. Source:Paterson&Hennessy Computer Organization&Design, 5 th Edition 14 7

8 Information retained if supply voltage is turned off Referred to as read-only memories (ROM), although some may be written to as well as read Read-only memory (ROM): programmed during production Programmable ROM (PROM): fuse associated with cell that is blown once by zapping with current; can be programmed once Eraseable PROM (EPROM): cells cleared by shining ultraviolet light, special device used to write 1 s; can be erased and reprogrammed about 1000 times Electrically eraseable PROM (EEPROM): similar to EPROM but does not require a physically separate programming device, can be re-programmed in place on printed circuit cards; can be reprogrammed about 100,000 times Flash Memory Based on EEPROM technology Nonvolatile Memory 15 Solid State Disk (SSD) Solid State Disk - SSD I/O bus Flash memory Block 0 Page 0 Page 1 Page P-1 Flash translation layer Requests to read and write logical disk blocks Block B-1 Page 0 Page 1 Page P-1 Pages: 51B to 4KB, Blocks: 3 to 18 pages Data read in units of pages. Page can be written only after its block has been erased A block wears out after 100,000 repeated writes. 16 8

9 SSD Characteristics Sequential read throughput 50 MB/s Sequential write throughput 170 MB/s Random read throughput 140 MB/s Random write throughput 14 MB/s Random read access 30 us Random write access 300 us Advantages of SSD over rotating disks: No moving parts semiconductor memory is more rugged Much faster random access times Use less power Disadvantages of SSD over rotating disks: SSDs wear out with usage 10-0 times more expensive than disks 17 Growth in Microprocessor Performance 18 9

10 Growth in Performance of RAM & CPU Mismatch between CPU performance growth and memory performance growth memory wall Importance of cache. 19 CPU Trends: Power Wall Inflection point in computer history when designers hit the Power Wall :1980 CPU Pentium P-III P-4 Core Core i7 --- Clock rate(mhz) Cycle time(ns) Cores Effective cycle ,000 time(ns) At about the same time, besides memory wall and power wall, processor designers also reached limits in taking advantage of instruction level parallelism ILP in (sequential) programs. Since early 000 s processors have not been (significantly) getting faster. Instead, multi-core processors. 0 10

11 Principle of Locality of Reference Programs access a small proportion of their address space at any time and they tend to reuse instructions and data they have used recently. temporal locality recently accessed items are likely to be accessed soon, spatial locality items near those accessed recently are likely to be accessed soon, i.e. items near one another tend to be referenced close together in time. An implication of principle of locality is that we can predict with reasonable accuracy what instructions and data a program will use in near future based on its accesses in the recent past. Principle of locality applies more strongly to code accesses than data accesses. 1 Taking Advantage of Locality Use memory hierarchy Store everything on disk Copy recently accessed (and nearby) items from disk to smaller DRAM memory Main memory (virtual memory) Copy recently accessed (and nearby) items from DRAM memory to smaller SRAM memory Cache attached to CPU 11

12 Typical System Organization Without Cache Register file CPU chip ALU System bus Memory bus Bus interface I/O bridge Main memory USB controller Graphics adapter I/O bus Disk controller Expansion slots for other devices such as network adapters. Mouse Keyboard Monitor Disk 3 Typical Processor Organization with Cache CPU chip Register file Cache memories ALU System bus Memory bus Bus interface I/O bridge Main memory Cache is fast but because of that it has to be small. Why? 4 1