Review: Assembly Programmer s View. The Memory Hierarchy. Random- Access Memory (RAM) Today. NonvolaHle Memories. SRAM vs DRAM Summary

Transcription

1 Review: ssembly Programmer s View The Hierarchy CSCI 1: Machine rchitecture and OrganizaHon Pen- Chung Yew Department Computer Science and Engineering University of Minnesota PC CPU Registers Condition Codes ddresses Data Instructions Object Code Program Data OS Data Stack With Slides from Bryant, O Hallaron and Zhai Today Storage technologies and trends Locality of reference Caching in the memory hierarchy Random- ccess (RM) Key features RM is tradihonally packaged as a chip. Basic storage unit is normally a cell (one bit per cell). MulHple RM chips form a memory. RM comes in two variehes: SRM (StaHc RM) DRM (Dynamic RM) SRM vs DRM Summary NonvolaHle Memories Trans. ccess Needs Needs per bit Hme refresh? EDC? Cost pplicahons SRM 4 or 6 1X No Maybe 1x Cache memories DRM 1 1X Yes Yes 1X Main memories, frame buffers DRM and SRM are volahle memories Lose informa,on if powered off. NonvolaHle memories retain value even if powered off Read- only memory (ROM): programmed during produc,on Programmable ROM (PROM): can be programmed once Eraseable PROM (EPROM): can be bulk erased (UV, X- Ray) Electrically eraseable PROM (EEPROM): electronic erase capability Flash memory: EEPROMs. with par,al (block- level) erase capability Wears out aker about 1, erasings Uses for NonvolaHle Memories Firmware programs stored in a ROM (BIOS, s for disks, network cards, graphics accelerators, security subsystems, ) Solid state disks (replace rota,ng disks in thumb drives, smart phones, mp3 players, tablets, laptops, ) caches 1

2 TradiHonal Bus Structure ConnecHng CPU and bus is a collechon of parallel wires that carry address, data, and control signals. Buses are typically shared by mulhple devices. CPU chip Read TransacHon (1) CPU places address on the memory bus. Load operahon: movq, %rax %rax I/O bridge x System bus bus I/O bridge Main memory Read TransacHon () reads from the memory bus, retrieves word x, and places it on the bus. Read TransacHon (3) CPU read word x from the bus and copies it into register %rax. Load operahon: movq, %rax Load operahon: movq, %rax %rax I/O bridge Main x memory %rax x I/O bridge x x Write TransacHon (1) CPU places address on bus. reads it and waits for the corresponding data word to arrive. Write TransacHon () CPU places data word y on the bus. Store operahon: movq %rax, Store operahon: movq %rax, %rax y %rax y I/O bridge I/O bridge y

3 Write TransacHon (3) What s Inside Drive? reads data word y from the bus and stores it at address. rm Spindle Platters Store operahon: movq %rax, ctuator %rax y I/O bridge main memory y SCSI connector Electronics (including a processor and memory!) Image courtesy of Seagate Technology Geometry s consist of pladers, each with two surfaces. Each surface consists of concentric rings called tracks. Geometry (Muliple- Plader View) ligned tracks form a cylinder. Cylinder k Each track consists of sectors separated by gaps. Tracks Surface Track k Gaps Surface Surface 1 Surface Surface 3 Surface 4 Surface 5 Plader Plader 1 Plader Spindle Spindle Sectors Capacity Capacity: maximum number of bits that can be stored. Vendors express capacity in units of gigabytes (GB), where 1 GB = 1 9 Bytes. Capacity is determined by these technology factors: Recording density (bits/in): number of bits that can be squeezed into a 1 inch segment of a track. Track density (tracks/in): number of tracks that can be squeezed into a 1 inch radial segment. real density (bits/in): product of recording and track density. Recording zones Modern disks par,,on tracks into disjoint subsets called recording zones Each track in a zone has the same number of sectors, determined by the circumference of innermost track. Each zone has a different number of sectors/track, outer zones have more sectors/track than inner zones. So we use average number of sectors/track when compu,ng capacity. Spindle 3

4 CompuHng Capacity OperaHon (Single- Plader View) Capacity = (# bytes/sector) x (avg. # sectors/track) x Example: (# tracks/surface) x (# surfaces/plader) x (# pladers/disk) 51 bytes/sector 3 sectors/track (on average), tracks/surface surfaces/plader 5 pladers/disk Capacity = 51 x 3 x x x 5 The disk surface spins at a fixed rotahonal rate spindle spindle spindle spindle The read/write head is adached to the end of the arm and flies over the disk surface on a thin cushion of air. By moving radially, the arm can posihon the read/write head over any track. = 3,7,, = 3.7 GB OperaHon (MulH- Plader View) Read/write heads move in unison from cylinder to cylinder rm Structure - top view of single plader Surface organized into tracks Tracks divided into sectors Spindle ccess ccess Head in position above a track Rotation is counter-clockwise 4

5 ccess Read ccess Read ker BLUE read bout to read blue sector fter reading blue sector ccess Read ccess Seek ker BLUE read ker BLUE read Seek for RED Red request scheduled next Seek to red s track ccess RotaHonal Latency ccess Read ker BLUE read Seek for RED RotaHonal latency ker BLUE read Seek for RED RotaHonal latency ker RED read Wait for red sector to rotate around Complete read of red 5

6 ccess Service Time Components ccess Time verage Hme to access some target sector approximated by : Taccess = Tavg seek + Tavg rotahon + Tavg transfer Seek Hme (Tavg seek) Time to posihon heads over cylinder containing target sector. Typical Tavg seek is 3 9 ms ker BLUE read Seek for RED RotaHonal latency ker RED read RotaHonal latency (Tavg rotahon) Time waihng for first bit of target sector to pass under r/w head. Tavg rotahon = 1/ x 1/RPMs x 6 sec/1 min Typical Tavg rotahon = 7 RPMs Data transfer Seek RotaHonal latency Data transfer Transfer Hme (Tavg transfer) Time to read the bits in the target sector. Tavg transfer = 1/RPM x 1/(avg # sectors/track) x 6 secs/1 min. ccess Time Example Logical Blocks Given: RotaHonal rate = 7, RPM verage seek Hme = 9 ms. vg # sectors/track = 4. Derived: Tavg rotahon = 1/ x (6 secs/7 RPM) x 1 ms/sec = 4 ms. Tavg transfer = 6/7 RPM x 1/4 secs/track x 1 ms/sec =. ms Taccess = 9 ms + 4 ms +. ms Important points: ccess,me dominated by seek,me and rota,onal latency. First bit in a sector is the most expensive, the rest are free. SRM access,me is about 4 ns/doubleword, DRM about 6 ns is about 4,,mes slower than SRM,,5,mes slower then DRM. Modern disks present a simpler abstract view of the complex sector geometry: The set of available sectors is modeled as a sequence of b- sized logical blocks (, 1,,...) Mapping between logical blocks and actual (physical) sectors Maintained by hardware/firmware device called disk. Converts requests for logical blocks into (surface,track,sector) triples. llows to set aside spare cylinders for each zone. ccounts for the difference in formaged capacity and maximum capacity. I/O Bus Reading a Sector (1) CPU chip System bus bus CPU chip CPU inihates a disk read by wrihng a command, logical block number, and deshnahon memory address to a port (address) associated with disk. I/O bridge Main memory Main memory I/O bus USB Graphics adapter I/O bus Expansion slots for other devices such as network adapters. USB Graphics adapter Mouse Keyboard Monitor mouse keyboard Monitor 6

7 Reading a Sector () Reading a Sector (3) CPU chip reads the sector and performs a direct memory access (DM) transfer into main memory. CPU chip When the DM transfer completes, the disk nohfies the CPU with an interrupt (i.e., asserts a special interrupt pin on the CPU) Main memory Main memory I/O bus I/O bus USB Graphics adapter USB Graphics adapter Mouse Keyboard Monitor Mouse Keyboard Monitor Solid State s (SSDs) Solid State (SSD) Flash memory Block Page Page 1 Page P-1 I/O bus Flash translation layer Pages: 51KB to 4KB, Blocks: 3 to 18 pages Data read/wrigen in units of pages. Requests to read and write logical disk blocks Block B-1 Page Page 1 Page P-1 Page can be wrigen only aker its block has been erased block wears out aker about 1, repeated writes. SSD Performance CharacterisHcs SequenHal read tput 55 MB/s SequenHal write tput 47 MB/s Random read tput 365 MB/s Random write tput 33 MB/s vg seq read Hme 5 us vg seq write Hme 6 us SequenHal access faster than random access Common theme in the memory hierarchy Random writes are somewhat slower Erasing a block takes a long Hme (~1 ms) Modifying a block page requires all other pages to be copied to new block In earlier SSDs, the read/write gap was much larger. Source: Intel SSD 73 product specificahon. SSD Tradeoffs vs RotaHng s The CPU- Gap dvantages No moving parts à faster, less power, more rugged Disadvantages Have the poten,al to wear out Mi,gated by wear leveling logic in flash transla,on layer E.g. Intel SSD 73 guarantees 18 petabyte (18 x 1 15 bytes) of writes before they wear out In 15, about 3,mes more expensive per byte pplicahons MP3 players, smart phones, laptops Beginning to appear in desktops and servers The gap Time (ns) 1,,. 1,,. 1,,. 1,. 1,. 1, between DRM, disk, and CPU speeds. SSD seek time SSD access time DRM access time SRM access time DRM CPU cycle time Effective CPU cycle time CPU Year 7

8 Locality to the Rescue! Today Storage technologies and trends The key to bridging this CPU- gap is a fundamental property of computer programs known as locality Locality of reference Caching in the memory hierarchy Locality Locality Example Principle of Locality: Programs tend to use data and instruchons with addresses near or equal to those they have used recently Temporal locality: Recently referenced items are likely to be referenced again in the near future SpaHal locality: Items with nearby addresses tend to be referenced close together in Hme Data references Reference array elements in succession (stride- 1 reference padern). Reference variable sum each iterahon. InstrucHon references sum = ; for (i = ; i < n; i++) sum += a[i]; return sum; Reference instruchons in sequence. Cycle through loop repeatedly. Spa,al locality Temporal locality Spa,al locality Temporal locality QualitaHve EsHmates of Locality Locality Example Claim: Being able to look at code and get a qualitahve sense of its locality is a key skill for a professional programmer. QuesHon: Does this funchon have good locality with respect to array a? QuesHon: Does this funchon have good locality with respect to array a? int sum_array_cols(int a[m][n]) { int i, j, sum = ; int sum_array_rows(int a[m][n]) { int i, j, sum = ; } for (i = ; i < M; i++) for (j = ; j < N; j++) sum += a[i][j]; return sum; } for (j = ; j < N; j++) for (i = ; i < M; i++) sum += a[i][j]; return sum; 8

9 Locality Example QuesHon: Can you permute the loops so that the funchon scans the 3- d array a with a stride- 1 reference padern (and thus has good spahal locality)? int sum_array_3d(int a[m][n][n]) { int i, j, k, sum = ; } for (i = ; i < M; i++) for (j = ; j < N; j++) for (k = ; k < N; k++) sum += a[k][i][j]; return sum; Hierarchies Some fundamental and enduring proper,es of hardware and sokware: Fast storage technologies cost more per byte, have less capacity, and require more power (heat!). The gap between CPU and main memory speed is widening. Well- wrigen programs tend to exhibit good locality. These fundamental proper,es complement each other beau,fully. They suggest an approach for organizing memory and storage systems known as a memory hierarchy. Today Storage technologies and trends Locality of reference Caching in the memory hierarchy Example Hierarchy Smaller, faster, and costlier (per byte) storage devices Larger, slower, and cheaper (per byte) storage L5: devices L6: L4: L3: L: L1: L: Regs L1 cache (SRM) L cache (SRM) L3 cache (SRM) (DRM) Local secondary storage (local disks) Remote secondary storage (e.g., Web servers) CPU registers hold words retrieved from the L1 cache. L1 cache holds cache lines retrieved from the L cache. L cache holds cache lines retrieved from L3 cache L3 cache holds cache lines retrieved from main memory. holds disk blocks retrieved from local disks. Local disks hold files retrieved from disks on remote servers Caches General Cache Concepts Cache: smaller, faster storage device that acts as a staging area for a subset of the data in a larger, slower device. Fundamental idea of a memory hierarchy: For each k, the faster, smaller device at level k serves as a cache for the larger, slower device at level k+1. Why do memory hierarchies work? Because of locality, programs tend to access the data at level k more oken than they access the data at level k+1. Thus, the storage at level k+1 can be slower, and thus larger and cheaper per bit. Big Idea: The memory hierarchy creates a large pool of storage that costs as much as the cheap storage near the bogom, but that serves data to programs at the rate of the fast storage near the top. Cache Data is copied in block- sized transfer units Smaller, faster, more expensive memory caches a subset of the blocks Larger, slower, cheaper memory viewed as parhhoned into blocks 9

10 General Cache Concepts: Hit General Cache Concepts: Miss Request: 14 Data in block b is needed Request: 1 Data in block b is needed Cache Block b is in cache: Hit! Cache Block b is not in cache: Miss! 1 Request: 1 Block b is fetched from memory Block b is stored in cache Placement policy: determines where b goes Replacement policy: determines which block gets evicted (vic,m) General Caching Concepts: Types of Cache Misses Cold (compulsory) miss Cold misses occur because the cache is empty. Conflict miss Most caches limit blocks at level k+1 to a small subset (some,mes a singleton) of the block posi,ons at level k. E.g. Block i at level k+1 must be placed in block (i mod 4) at level k. Conflict misses occur when the level k cache is large enough, but mul,ple data objects all map to the same level k block. E.g. Referencing blocks, 8,, 8,, 8,... would miss every,me. Capacity miss Occurs when the set of ac,ve cache blocks (working set) is larger than the cache. Examples of Caching in the Mem. Hierarchy Cache Type Registers TLB L1 cache L cache Virtual Buffer cache Network buffer cache Browser cache What is Cached? 4-8 bytes words ddress transla,ons 64- byte blocks 64- byte blocks 4- KB pages Parts of files Parts of files Web pages Where is it Cached? CPU core On- Chip TLB On- Chip L1 On- Chip L Local disk Local disk Latency (cycles) ,, 1,, Managed By Compiler Hardware MMU Hardware Hardware Hardware + OS cache sectors 1, firmware OS NFS client Web browser Web cache Web pages Remote server disks 1,,, Web proxy server Summary Supplemental slides The speed gap between CPU, memory and mass storage conhnues to widen. Well- wriden programs exhibit a property called locality. hierarchies based on caching close the gap by exploihng locality. 1

11 ConvenHonal DRM OrganizaHon d x w DRM: dw total bits organized as d supercells of size w bits Reading DRM Supercell (,1) Step 1(a): Row access strobe (RS) selects row. Step 1(b): Row copied from DRM array to row buffer. 16 x 8 DRM chip 16 x 8 DRM chip (to/from CPU) bits / addr 8 bits / data 1 rows 3 cols 1 3 supercell (,1) RS = / addr 8 / data 1 Rows 3 Cols 1 3 Internal row buffer Internal row buffer Reading DRM Supercell (,1) Step (a): Column access strobe (CS) selects column 1. Step (b): Supercell (,1) copied from buffer to data lines, and eventually back to the CPU. To CPU supercell (,1) CS = 1 / addr 8 / data 16 x 8 DRM chip 1 Rows 3 Cols 1 3 Modules addr (row = i, col = j) DRM DRM 7 bits bits bits bits bits bits bits bits bit word main memory address : supercell (i,j) 64 MB memory module consishng of eight 8Mx8 DRMs supercell (,1) Internal row buffer 64- bit word Enhanced DRMs Basic DRM cell has not changed since its invenhon in Commercialized by Intel in 197. DRM cores with beder interface logic and faster I/O : Synchronous DRM (SDRM) Uses a conven,onal clock signal instead of asynchronous control llows reuse of the row addresses (e.g., RS, CS, CS, CS) Double data- rate synchronous DRM (DDR SDRM) Double edge clocking sends two bits per cycle per pin Different types dis,nguished by size of small prefetch buffer: DDR ( bits), DDR (4 bits), DDR3 (8 bits) By 1, standard for most server and desktop systems Intel Core i7 supports only DDR3 SDRM Storage Trends SRM Metric :1985 $/MB, access (ns) DRM Metric :1985 $/MB , access (ns) typical size (MB) , 8, 16. 6,5 Metric :1985 $/GB 1, 8, ,333,333 access (ms) typical size (GB) ,5 3, 3, 11

12 CPU Clock Rates InflecHon point in computer history when designers hit the Power Wall :1985 CPU Pen,um P- 4 Core Core i7(n) Core i7(h) Clock rate (MHz) ,3,,5 3, 5 Cycle,me (ns) Cores Effec,ve cycle ,75,me (ns) (n) Nehalem processor (h) Haswell processor 1