Today. The Memory Hierarchy. Random-Access Memory (RAM) Nonvolatile Memories. Traditional Bus Structure Connecting CPU and Memory

Transcription

1 Today The Hierarchy Storage technologies and trends Locality of reference Caching in the hierarchy CSci 1: Machine rchitecture and Organization November 5th-7th, 18 Your instructor: Stephen McCamant Based on slides originally by: Randy Bryant, Dave O Hallaron 1 Random-ccess (RM) SRM vs DRM Summary Key features RM is traditionally packaged as a chip. Basic storage unit is normally a cell (one bit per cell). Multiple RM chips form a. Trans. ccess Needs Needs per bit time refresh? EDC? Cost pplications RM comes in two varieties: SRM (Static RM) DRM (Dynamic RM) SRM 4 or 6 1X No Maybe 1x Cache memories DRM 1 1X Yes Yes 1X Main memories, frame buffers 3 4 Nonvolatile Memories DRM and SRM are volatile memories Lose information if powered off. Nonvolatile memories retain value even if powered off Read-only (ROM): programmed during production Programmable ROM (PROM): can be programmed once Eraseable PROM (EPROM): can be bulk erased (UV, X-Ray) Electrically eraseable PROM (EEPROM): electronic erase capability Flash : EEPROMs. with partial (block-level) erase capability Wears out after about 1, erasings Uses for Nonvolatile Memories Firmware programs stored in a ROM (BIOS, s for disks, network cards, graphics accelerators, security subsystems, ) Solid state disks (replace rotating disks in thumb drives, smart phones, mp3 players, tablets, laptops, ) caches Traditional Bus Structure Connecting CPU and bus is a collection of parallel wires that carry address, data, and control signals. Buses are typically shared by multiple devices. CPU chip System bus IO bridge bus Main 5 6 1

2 Read Transaction (1) CPU places address on the bus. Read Transaction () Main reads from the bus, retrieves word x, and places it on the bus. Load operation: movq, Load operation: movq, Main x Main x x 7 8 Read Transaction (3) Write Transaction (1) CPU read word x from the bus and copies it into register. CPU places address on bus. Main reads it and waits for the corresponding data word to arrive. Load operation: movq, Store operation: movq, x y Main x Main 9 1 Write Transaction () Write Transaction (3) CPU places data word y on the bus. Main reads data word y from the bus and stores it at address. Store operation: movq, Store operation: movq, y y y Main main y 11 1

3 What s Inside Drive? Geometry ctuator rm Spindle Platters s consist of platters, each with two surfaces. Each surface consists of concentric rings called tracks. Each track consists of sectors separated by gaps. Tracks Surface Track k Gaps SCSI connector Electronics (including a processor and!) Spindle Image courtesy of Seagate Technology Sectors Geometry (Muliple-Platter View) Capacity ligned tracks form a cylinder. Surface Surface 1 Surface Surface 3 Surface 4 Surface 5 Cylinder k Platter Platter 1 Platter Capacity: maximum number of bits that can be stored. Vendors express capacity in units of gigabytes (GB), where 1 GB = 1 9 Bytes. Capacity is determined by these technology factors: Recording density (bitsin): number of bits that can be squeezed into a 1 inch segment of a track. Track density (tracksin): number of tracks that can be squeezed into a 1 inch radial segment. real density (bitsin): product of recording and track density. Spindle Recording zones Computing Capacity Modern disks partition tracks into disjoint subsets called recording zones Each track in a zone has the same number of sectors, determined by the circumference of innermost track. Each zone has a different number of sectorstrack, outer zones have more sectorstrack than inner zones. So we use average number of sectorstrack when computing capacity. Spindle Capacity = (# bytessector) x (avg. # sectorstrack) x (# trackssurface) x (# surfacesplatter) x (# plattersdisk) Example: 51 bytessector 3 sectorstrack (on average), trackssurface surfacesplatter 5 plattersdisk Capacity = 51 x 3 x x x 5 = 3,7,, = 3.7 GB

4 spindle Operation (Single-Platter View) Operation (Multi-Platter View) The disk surface spins at a fixed rotational rate The readwrite head is attached to the end of the arm and flies over the disk surface on a thin cushion of air. Readwrite heads move in unison from cylinder to cylinder rm spindle spindle spindle Spindle By moving radially, the arm can position the readwrite head over any track. 19 Structure - top view of single platter ccess Surface organized into tracks Tracks divided into sectors Head in position above a track 1 ccess ccess Read Rotation is counter-clockwise bout to read blue sector 3 4 4

5 ccess Read ccess Read fter BLUE read fter BLUE read fter reading blue sector Red request scheduled next 5 6 ccess Seek ccess Rotational Latency fter BLUE read Seek for RED fter BLUE read Seek for RED Rotational latency Seek to red s track Wait for red sector to rotate around 7 8 ccess Read ccess Service Time Components fter BLUE read Seek for RED Rotational latency fter RED read fter BLUE read Seek for RED Rotational latency fter RED read Complete read of red Data transfer Seek Rotational latency Data transfer 9 3 5

6 ccess Time ccess Time Example verage time to access some target sector approximated by : Taccess = Tavg seek + Tavg rotation + Tavg transfer Seek time (Tavg seek) Time to position heads over cylinder containing target sector. Typical Tavg seek is 3 9 ms Rotational latency (Tavg rotation) Time waiting for first bit of target sector to pass under rw head. Tavg rotation = 1 x 1RPMs x 6 sec1 min Typical Tavg rotation = 7 RPMs Transfer time (Tavg transfer) Time to read the bits in the target sector. Tavg transfer = 1RPM x 1(avg # sectorstrack) x 6 secs1 min. 31 Given: Rotational rate = 7, RPM verage seek time = 9 ms. vg # sectorstrack = 4. Derived: Tavg rotation = 1 x (6 secs7 RPM) x 1 mssec = 4 ms. Tavg transfer = 67 RPM x 14 secstrack x 1 mssec =. ms Taccess = 9 ms + 4 ms +. ms Important points: ccess time dominated by seek time and rotational latency. First bit in a sector is the most expensive, the rest are free. SRM access time is about 4 nsdoubleword, DRM about 6 ns is about 4, times slower than SRM,,5 times slower then DRM. 3 Logical Blocks IO Bus CPU chip Modern disks present a simpler abstract view of the complex sector geometry: The set of available sectors is modeled as a sequence of b-sized logical blocks (, 1,,...) Mapping between logical blocks and actual (physical) sectors Maintained by hardwarefirmware device called disk. Converts requests for logical blocks into (surface,track,sector) triples. llows to set aside spare cylinders for each zone. ccounts for the difference in formatted capacity and maximum capacity. USB Graphics adapter Mouse Keyboard Monitor System bus IO bridge IO bus bus Main Expansion slots for other devices such as network adapters Reading a Sector (1) Reading a Sector () CPU chip CPU initiates a disk read by writing a command, logical block number, and destination address to a port (address) associated with disk. CPU chip reads the sector and performs a direct access (DM) transfer into main. Main Main IO bus IO bus USB Graphics adapter USB Graphics adapter mouse keyboard Monitor Mouse Keyboard Monitor

7 Time (ns) Reading a Sector (3) CPU chip When the DM transfer completes, the disk notifies the CPU with an interrupt (i.e., asserts a special interrupt pin on the CPU) IO bus Main Solid State s (SSDs) Solid State (SSD) IO bus Flash Block Page Page 1 Page P-1 Flash translation layer Requests to read and write logical disk blocks Block B-1 Page Page 1 Page P-1 Pages: 51B to 4KB, Blocks: 3 to 18 pages USB Graphics adapter Mouse Keyboard Monitor 37 Data readwritten in units of pages. Page can be written only after its block has been erased block wears out after about 1, repeated writes. 38 SSD Performance Characteristics SSD Tradeoffs vs Rotating s Sequential read tput 55 MBs Sequential write tput 47 MBs Random read tput 365 MBs Random write tput 33 MBs vg seq read time 5 us vg seq write time 6 us Sequential access faster than random access Common theme in the hierarchy Random writes are somewhat slower Erasing a block takes a long time (~1 ms) Modifying a block page requires all other pages to be copied to new block In earlier SSDs, the readwrite gap was much larger. Source: Intel SSD 73 product specification. 39 dvantages No moving parts faster, less power, more rugged Disadvantages Have the potential to wear out Mitigated by wear leveling logic in flash translation layer E.g. Intel SSD 73 guarantees 18 petabyte (18 x 1 15 bytes) of writes before they wear out In 15, about 3 times more expensive per byte pplications MP3 players, smart phones, laptops Beginning to appear in desktops and servers 4 The CPU- Gap The gap between DRM, disk, and CPU speeds. Locality to the Rescue! 1,,. 1,,. 1,,. 1,. SSD The key to bridging this CPU- gap is a fundamental property of computer programs known as locality 1,. 1,. seek time SSD access time DRM access time 1. DRM SRM access time CPU cycle time 1. Effective CPU cycle time 1..1 CPU Year

8 Today Storage technologies and trends Locality of reference Caching in the hierarchy Locality Principle of Locality: Programs tend to use data and instructions with addresses near or equal to those they have used recently Temporal locality: Recently referenced items are likely to be referenced again in the near future Spatial locality: Items with nearby addresses tend to be referenced close together in time Locality Example Qualitative Estimates of Locality sum = ; for (i = ; i < n; i++) sum += a[i]; return sum; Claim: Being able to look at code and get a qualitative sense of its locality is a key skill for a professional programmer. Data references Reference array elements in succession (stride-1 reference pattern). Reference variable sum each iteration. Instruction references Reference instructions in sequence. Cycle through loop repeatedly. Spatial locality Temporal locality Spatial locality Temporal locality Question: Does this function have good locality with respect to array a? int sum_array_rows(int a[m][n]) { int i, j, sum = ; } for (i = ; i < M; i++) for (j = ; j < N; j++) sum += a[i][j]; return sum; Locality Example Locality Example Question: Does this function have good locality with respect to array a? int sum_array_cols(int a[m][n]) { int i, j, sum = ; } for (j = ; j < N; j++) for (i = ; i < M; i++) sum += a[i][j]; return sum; Question: Can you permute the loops so that the function scans the 3-d array a with a stride-1 reference pattern (and thus has good spatial locality)? int sum_array_3d(int a[m][n][n]) { int i, j, k, sum = ; } for (i = ; i < M; i++) for (j = ; j < N; j++) for (k = ; k < N; k++) sum += a[k][i][j]; return sum;

9 Hierarchies Some fundamental and enduring properties of hardware and software: Fast storage technologies cost more per byte, have less capacity, and require more power (heat!). The gap between CPU and main speed is widening. Well-written programs tend to exhibit good locality. Today Storage technologies and trends Locality of reference Caching in the hierarchy These fundamental properties complement each other beautifully. They suggest an approach for organizing and storage systems known as a hierarchy Example Hierarchy Smaller, faster, and costlier (per byte) storage devices Larger, slower, and cheaper (per byte) storage L5: devices L6: L4: L3: L: L1: L: Regs L1 cache (SRM) L cache (SRM) L3 cache (SRM) Main (DRM) Local secondary storage (local disks) Remote secondary storage (e.g., Web servers) CPU registers hold words retrieved from the L1 cache. L1 cache holds cache lines retrieved from the L cache. L cache holds cache lines retrieved from L3 cache L3 cache holds cache lines retrieved from main. Main holds disk blocks retrieved from local disks. Local disks hold files retrieved from disks on remote servers 51 Caches Cache: smaller, faster storage device that acts as a staging area for a subset of the data in a larger, slower device. Fundamental idea of a hierarchy: For each k, the faster, smaller device at level k serves as a cache for the larger, slower device at level k+1. Why do hierarchies work? Because of locality, programs tend to access the data at level k more often than they access the data at level k+1. Thus, the storage at level k+1 can be slower, and thus larger and cheaper per bit. Big Idea: The hierarchy creates a large pool of storage that costs as much as the cheap storage near the bottom, but that serves data to programs at the rate of the fast storage near the top. 5 General Cache Concepts General Cache Concepts: Hit Request: 14 Data in block b is needed Cache Smaller, faster, more expensive caches a subset of the blocks Cache Block b is in cache: Hit! 1 4 Data is copied in block-sized transfer units Larger, slower, cheaper viewed as partitioned into blocks

10 General Cache Concepts: Miss General Caching Concepts: Types of Cache Misses Cache Request: Request: Data in block b is needed Block b is not in cache: Miss! Block b is fetched from Block b is stored in cache Placement policy: determines where b goes Replacement policy: determines which block gets evicted (victim) Cold (compulsory) miss Cold misses occur because the cache is empty. Conflict miss Most caches limit blocks at level k+1 to a small subset (sometimes a singleton) of the block positions at level k. E.g. Block i at level k+1 must be placed in block (i mod 4) at level k. Conflict misses occur when the level k cache is large enough, but multiple data objects all map to the same level k block. E.g. Referencing blocks, 8,, 8,, 8,... would miss every time. Capacity miss Occurs when the set of active cache blocks (working set) is larger than the cache Examples of Caching in the Mem. Hierarchy Summary Cache Type Registers TLB L1 cache L cache Virtual Buffer cache Network buffer cache Browser cache What is Cached? 4-8 bytes words ddress translations 64-byte blocks 64-byte blocks 4-KB pages Parts of files Parts of files Web pages Where is it Cached? CPU core On-Chip TLB On-Chip L1 On-Chip L Main Main Local disk Local disk Latency (cycles) ,, 1,, Managed By Compiler Hardware MMU Hardware Hardware Hardware + OS cache sectors 1, firmware OS NFS client Web browser The speed gap between CPU, and mass storage continues to widen. Well-written programs exhibit a property called locality. hierarchies based on caching close the gap by exploiting locality. Web cache Web pages Remote server disks 1,,, Web proxy server Supplemental slides Conventional DRM Organization d x w DRM: dw total bits organized as d supercells of size w bits 16 x 8 DRM chip cols 1 3 (tofrom CPU) bits addr 8 bits data rows 1 3 supercell (,1) Internal row buffer

11 Reading DRM Supercell (,1) Step 1(a): Row access strobe (RS) selects row. Step 1(b): Row copied from DRM array to row buffer. RS = addr 8 data 16 x 8 DRM chip 1 Rows 3 Cols 1 3 Reading DRM Supercell (,1) Step (a): Column access strobe (CS) selects column 1. Step (b): Supercell (,1) copied from buffer to data lines, and eventually back to the CPU. To CPU supercell (,1) CS = 1 addr 8 data 16 x 8 DRM chip 1 Rows 3 Cols 1 3 Internal row buffer 61 supercell (,1) Internal row buffer 6 Modules Enhanced DRMs addr (row = i, col = j) DRM 7 DRM : supercell (i,j) 64 MB module consisting of eight 8Mx8 DRMs Basic DRM cell has not changed since its invention in Commercialized by Intel in 197. DRM cores with better interface logic and faster IO : Synchronous DRM (SDRM) Uses a conventional clock signal instead of asynchronous control llows reuse of the row addresses (e.g., RS, CS, CS, CS) bits bits bits bits bits bits bits bit word main address 64-bit word bits -7 Double data-rate synchronous DRM (DDR SDRM) Double edge clocking sends two bits per cycle per pin Different types distinguished by size of small prefetch buffer: DDR ( bits), DDR (4 bits), DDR3 (8 bits) By 1, standard for most server and desktop systems Intel Core i7 supports only DDR3 SDRM Storage Trends SRM Metric :1985 $MB, access (ns) DRM Metric :1985 $MB , access (ns) typical size (MB) , 8, 16. 6,5 Metric :1985 $GB 1, 8, ,333,333 access (ms) typical size (GB) ,5 3, 3, 65 CPU Clock Rates Inflection point in computer history when designers hit the Power Wall :1985 CPU Pentium P-4 Core Core i7(n) Core i7(h) Clock rate (MHz) ,3,,5 3, 5 Cycle time (ns) Cores Effective cycle ,75 time (ns) (n) Nehalem processor (h) Haswell processor 66 11