Announcements. Outline

Transcription

1 15-13 The course that gives CMU its Zip! The Memory Hierarchy Feb. 14, 8 Topics Storage technologies and trends Locality of reference Caching in the hierarchy nnouncements Recitation room changes C (Nate) Doherty 111 G (Pratyusa) Porter H (lly) Porter 19 Exam date change NOT Thursday, 1 CHNGED TO Tuesday, 6: 7: p.m. - 8:3 p.m. UC McConomy XOR Wean 75 (expect ) Calculator policy Calculators will not be needed on the exam; hence forbidden. Collaboration reminder Writing code together counts as sharing code - forbidden Talking through a problem can include pictures (not code) class1.ppt 15-13, S 8 Opinion Poll Plan 14 Thu Memory Hierarchy (DE) 19 Tue Opt II (REB) 1 Thu No class? [Bill Gates] 6 Tue Cache Memories (DE) [Evening Exam] 8 Thu Linking (DE) Back to original schedule Plan B 14 Thu Memory Hierarchy (DE) 19 Tue Opt II (REB) 1 Thu Cache Memories (DE) [Bill Gates] 6 Tue No class? [Evening Exam] 8 Thu Linking (DE) Back to original schedule Outline RM ROM Disks Mind the gap! Locality Memory Hierarchy Caches , S , S 8 Random-ccess Memory (RM) Key features RM is traditionally packaged as a chip. Basic storage unit is normally a cell (one bit per cell). Multiple RM chips form a. Static RM (SRM( SRM) Each cell stores a bit with a four or six-transistor circuit. Retains value indefinitely, as long as it is kept powered. Relatively insensitive to electrical noise (EMI), radiation, etc. Faster and more expensive than DRM. Dynamic RM (DRM( DRM) Each cell stores bit with a capacitor. One transistor is used for access Value must be refreshed every 1-1 ms. More sensitive to disturbances (EMI, radiation, ) than SRM. Slower and cheaper than SRM , S 8 SRM vs DRM Summary Tran. ccess Needs Needs per bit time refresh? EDC? Cost pplications SRM 4 or 6 1X No Maybe 1x cache memories DRM 1 1X Yes Yes 1X Main memories, frame buffers , S 8

2 Conventional DRM Organization d x w DRM: dw total bits organized as d supercells of size w bits 16 x 8 DRM chip Reading DRM Supercell (,1) Step 1(a): Row access strobe (RS( RS) ) selects row. Step 1(b): Row copied from DRM array to row buffer. 16 x 8 DRM chip (to CPU) bits addr 8 bits data 1 rows 3 cols 1 3 supercell (,1) RS = addr 8 data 1 rows 3 cols 1 3 internal row buffer , S 8 internal row buffer , S 8 Reading DRM Supercell (,1) Step (a): Column access strobe (CS( CS) ) selects column 1. Step (b): Supercell (,1) copied from buffer to data lines, and eventually back to the CPU. To CPU supercell (,1) CS = 1 addr 8 data 16 x 8 DRM chip rows 1 3 cols 1 3 Memory Modules addr (row = i, col = j) : supercell (i,j) DRM 64 MB module consisting of DRM 7 eight 8Mx8 DRMs bits bits bits bits bits bits bits bits Memory 64-bit doubleword at address supercell 9 internal row buffer (,1) 15-13, S 8 64-bit doubleword , S 8 Enhanced DRMs DRM Cores with better interface logic and faster IO : Synchronous DRM (SDRM) Uses a conventional clock signal instead of asynchronous control Double data-rate synchronous DRM (DDR SDRM) Double edge clocking sends two bits per cycle per pin RamBus DRM (RDRM) Uses faster signaling over fewer wires (source directed clocking) with a Transaction oriented interface protocol Obsolete Technologies : Fast page mode DRM (FPM DRM) llowed re-use of row-addresses Extended data out DRM (EDO DRM) Enhanced FPM DRM with more closely spaced CS signals. Video RM (VRM) Dual ported FPM DRM with a second, concurrent, serial interface Extra functionality DRMS (CDRM, GDRM) dded SRM (CDRM) and support for graphics operations (GDRM) , S 8 Nonvolatile Memories DRM and SRM are volatile memories Lose information if powered off. Nonvolatile memories retain value even if powered off Read-only (ROM): programmed during production Magnetic RM (MRM): stores bit magnetically (in development) Ferro-electric RM (FERM): uses a ferro-electric dielectric Programmable ROM (PROM): can be programmed once Eraseable PROM (EPROM): can be bulk erased (UV, X-Ray) Electrically eraseable PROM (EEPROM): electronic erase capability Flash : EEPROMs with partial (sector) erase capability Uses for Nonvolatile Memories Firmware programs stored in a ROM (BIOS, s for s, network cards, graphics accelerators, security subsystems, ) Solid state s (flash cards, sticks, etc.) Smart cards, embedded systems, appliances Disk caches , S 8

3 Traditional Bus Structure Connecting CPU and Memory bus is a collection of parallel wires that carry address, data, and control signals. Buses are typically shared by multiple devices. Memory Read Transaction (1) CPU places address on the bus. Load operation: movl, x system bus bus IO bridge , S , S 8 Memory Read Transaction () Main reads from the bus, retrieves word x, and places it on the bus. Memory Read Transaction (3) CPU read word x from the bus and copies it into register. Load operation: movl, Load operation: movl, x x x x , S , S 8 Memory Write Transaction (1) CPU places address on bus. Main reads it and waits for the corresponding data word to arrive. Memory Write Transaction () CPU places data word y on the bus. Store operation: movl, Store operation: movl, y y y , S , S 8

4 Memory Write Transaction (3) Main reads data word y from the bus and stores it at address. Store operation: movl, y y Memory Subsystem Trends Observation: DRM chip has an access time of about 5ns. Traditional systems may need 3x longer to get the data from into a CPU register. Modern systems integrate the onto the : Latency matters! DRM and SRM densities increase and so does the soft-error rate: Traditional error detection & correction (EDC) is a must have (64bit of data plus 8bits of redundancy allow any 1 bit error to be corrected and any bit error is guaranteed to be detected) EDC is increasingly needed for SRMs too ChipKill capability (can correct all bits supplied by one failing chip) will become standard soon , S , S 8 Disk Geometry Disks consist of platters,, each with two surfaces. Each surface consists of concentric rings called tracks. Each track consists of sectors separated by gaps. tracks surface track k gaps Disk Geometry (Muliple-Platter View) ligned tracks form a cylinder. surface surface 1 surface surface 3 surface 4 surface 5 cylinder k platter platter 1 platter sectors , S , S 8 Disk Capacity Capacity: maximum number of bits that can be stored. Vendors express capacity in units of gigabytes (GB), where 1 GB = 1 9 Bytes (Lawsuit pending! Claims deceptive advertising). Sector Sizes Sectors on the outer tracks appear larger More area per bit might be more reliable (for precious data?) s number of tracks gets large, this could get wasteful Mathematical fact or optical illusion? Capacity is determined by these technology factors: Recording density (bitsin): number of bits that can be squeezed into a 1 inch segment of a track. Track density (tracksin): number of tracks that can be squeezed into a 1 inch radial segment. real density (bitsin ): product of recording and track density , S 8 Taken from some CMU ECE lecture (thanks!) , S 8

5 Zoned Bit Recording (ZBR) Modern s partition tracks into disjoint subsets called recording zones Each track in a zone has the same number of sectors, determined by the circumference of innermost track. Each zone has a different number of sectorstrack Computing Disk Capacity Capacity = (# bytessector) x (avg. # sectorstrack) x (# trackssurface) x (# surfacesplatter) x (# platters) Example: 51 bytessector 3 sectorstrack (on average), trackssurface surfacesplatter 5 platters Taken from Reference Guide Hard Disk Drives , S 8 Capacity = 51 x 3 x x x 5 = 3,7,, = 3.7 GB , S 8 Disk Operation (Single-Platter View) Disk Operation (Multi-Platter View) The surface spins at a fixed rotational rate The readwrite head is attached to the end of the arm and flies over the surface on a thin cushion of air. readwrite heads move in unison from cylinder to cylinder arm By moving radially, the arm can position the readwrite head over any track , S , S 8 Disk ccess Time verage time to access some target sector approximated by : Taccess = Tavg seek + Tavg rotation + Tavg transfer Seek time (Tavg seek) Time to position heads over cylinder containing target sector. Typical Tavg seek = 9 ms Rotational latency (Tavg rotation) Time waiting for first bit of target sector to pass under rw head. Tavg rotation = 1 x 1RPMs x 6 sec1 min Transfer time (Tavg transfer) Time to read the bits in the target sector. Tavg transfer = 1RPM x 1(avg # sectorstrack) x 6 secs1 min , S 8 Disk ccess Time Example Given: Rotational rate = 7, RPM verage seek time = 9 ms. vg # sectorstrack = 4. Derived: Tavg rotation = 1 x (6 secs7 RPM) x 1 mssec = 4 ms. Tavg transfer = 67 RPM x 14 secstrack x 1 mssec =. ms Taccess = 9 ms + 4 ms +. ms Important points: ccess time dominated by seek time and rotational latency. First bit in a sector is the most expensive, the rest are free. SRM access time is about 4 nsdoubleword, DRM about 6 ns Disk is about 4, times slower than SRM,,5 times slower then DRM , S 8

6 Logical Disk Blocks Modern s present a simpler abstract view of the complex sector geometry: The set of available sectors is modeled as a sequence of b- sized logical blocks (, 1,,...) Mapping between logical blocks and actual (physical) sectors Maintained by hardwarefirmware device called. Converts requests for logical blocks into (surface,track,sector) triples. This approach is logical block addressing ( LB ) llows to set aside spare cylinders for each zone. ccounts for the difference in formatted capacity and maximum capacity , S 8 IO Bus system bus bus IO bridge IO bus Expansion slots for other devices such USB graphics as network adapters. adapter mousekeyboard monitor , S 8 Reading a Disk Sector (1) CPU initiates a read by writing a command, logical block number, and destination address to a port (address) associated with. Reading a Disk Sector () Disk reads the sector and performs a direct access (DM) transfer into. IO bus IO bus USB graphics adapter USB graphics adapter mousekeyboard monitor , S 8 mousekeyboard monitor , S 8 Reading a Disk Sector (3) When the DM transfer completes, the notifies the CPU with an interrupt (i.e., asserts a special interrupt pin on the CPU) Storage Trends SRM metric :198 $MB 19,, access (ns) DRM metric :198 IO bus USB graphics adapter mousekeyboard monitor , S 8 $MB 8, , access (ns) typical size(mb) , 15, Disk metric :198 $MB , access (ms) typical size(mb) , 9, 4, 4, , S 8

7 CPU Clock Rates The CPU-Memory Gap The gap widens between DRM,, and CPU speeds : Pentium P-III P-4 processor clock rate(mhz) , 3, cycle time(ns) 1, , , S 8 1,, 1,, 1,, 1, 1, 1, 1 ns , S 8 Year Disk seek time DRM access time SRM access time CPU cycle time Locality Principle of Locality: Programs tend to reuse data and instructions near those they have used recently, or that were recently referenced themselves. Temporal locality: Recently referenced items are likely to be referenced in the near future. Spatial locality: Items with nearby addresses tend to be referenced close together in time. Locality Example: Data sum = ; for (i = ; i < n; i++) sum += a[i]; Reference array elements in succession return sum; (stride-1 reference pattern): Spatial locality Reference sum each iteration: Temporal locality Instructions Reference instructions in sequence: Spatial locality Cycle through loop repeatedly: Temporal locality , S 8 Locality Example Claim: Being able to look at code and get a qualitative sense of its locality is a key skill for a professional programmer. Question: Does this function have good locality? int sum_array_rows(int a[m][n]) { int i, j, sum = ; } for (i = ; i < M; i++) for (j = ; j < N; j++) sum += a[i][j]; return sum; , S 8 Locality Example Question: Does this function have good locality? int sum_array_cols(int a[m][n]) { int i, j, sum = ; } for (j = ; j < N; j++) for (i = ; i < M; i++) sum += a[i][j]; return sum; Locality Example Question: Can you permute the loops so that the function scans the 3-d array a[] with a stride-1 reference pattern (and thus has good spatial locality)? int sum_array_3d(int a[m][n][n]) { int i, j, k, sum = ; } for (i = ; i < M; i++) for (j = ; j < N; j++) for (k = ; k < N; k++) sum += a[k][i][j]; return sum; , S , S 8

8 Memory Hierarchies n Example Memory Hierarchy Some fundamental and enduring properties of hardware and software: Fast storage technologies cost more per byte, have less capacity, and require more power (heat!). The gap between CPU and speed is widening. Well-written programs tend to exhibit good locality. These fundamental properties complement each other beautifully. They suggest an approach for organizing and storage systems known as a hierarchy. Smaller, faster, and costlier (per byte) storage devices Larger, slower, and cheaper (per byte) storage devices L5: L: registers CPU registers hold words retrieved from L1 cache. L1: on-chip L1 cache (SRM) L1 cache holds cache lines retrieved from the L cache. L: off-chip L cache (SRM) L cache holds cache lines retrieved from. L3: (DRM) Main holds blocks retrieved from local s. L4: local secondary storage (local s) Local s hold files retrieved from s on remote network servers. remote secondary storage (tapes, distributed file systems, Web servers) , S , S 8 Caches Cache: smaller, faster storage device that acts as a staging area for a subset of the data in a larger, slower device. Fundamental idea of a hierarchy: For each k, the faster, smaller device at level k serves as a cache for the larger, slower device at level k+1. Why do hierarchies work? Programs tend to access the data at level k more often than they access the data at level k+1. Thus, the storage at level k+1 can be slower, and thus larger and cheaper per bit. Net effect: large pool of that costs as much as the cheap storage near the bottom, but that serves data to programs at the rate of the fast storage near the top. Caching in a Memory Hierarchy Smaller, faster, more expensive Level k: device at level k caches a subset of the blocks from level k+1 Data is copied between levels in block-sized transfer 1 4 units Larger, slower, cheaper storage Level k+1: device at level k+1 is partitioned into blocks , S , S 8 General Caching Concepts General Caching Concepts Level k: Level k+1: 14 1 Request * * Request * Program needs object d, which is stored in some block b. Cache hit Program finds b in the cache at level k. E.g., block 14. Cache miss b is not at level k, so level k cache must fetch it from level k+1. E.g., block 1. If level k cache is full, then some current block must be replaced (evicted). Which one is the victim? Placement policy: where can the new block go? E.g., b mod 4 Replacement policy: which block should be evicted? E.g., LRU Types of cache misses: Cold (compulsory) miss Cold misses occur because the cache is empty. Conflict miss Most caches limit blocks at level k+1 to a small subset (sometimes a singleton) of the block positions at level k. E.g. Block i at level k+1 must be placed in block (i mod 4) at level k+1. Conflict misses occur when the level k cache is large enough, but multiple data objects all map to the same level k block. E.g. Referencing blocks, 8,, 8,, 8,... would miss every time. Capacity miss Occurs when the set of active cache blocks (working set) is larger than the cache , S , S 8

9 Examples of Caching in the Hierarchy Cache Type Registers TLB L1 cache L cache Virtual Memory Buffer cache Network buffer Parts of files cache Browser cache Web pages Web cache What is Cached? 4-byte words ddress On-Chip TLB translations 64-bytes block On-Chip L1 64-bytes block Off-Chip L 4-KB page Main Parts of files Web pages Where is it Cached? CPU core Main Local Local Remote server s Latency (cycles) Managed By Compiler Hardware 1 Hardware 1 Hardware 1 Hardware+ OS 1 OS 1,, FSNFS client 1,, Web browser 1,,, Web proxy server , S 8 Summary The hierarchy is fundamental consequence of taining the random access abstraction and practical limits on cost and power consumption. Caching works! Programming for good temporal and spatial locality is critical for high performance. Trend: the speed gap between CPU, and mass storage continues to widen, thus leading towards deeper hierarchies. Consequence: taining locality becomes even more important , S 8