Computer Organization: A Programmer's Perspective

A Programmer's Perspective Computer Architecture and The Memory Hierarchy Gal A. Kaminka galk@cs.biu.ac.il

Typical Computer Architecture CPU chip PC (Program Counter) register file ALU Main Components CPU Main Memory Input/Output Devices Buses bus interface I/O Bridge main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 2

Typical Computer Architecture CPU chip PC (Program Counter) register file ALU CPU Algorithm: Read instruction from address in PC Interpret bits in instruction Perform instruction using ALU Update PC (sometimes based on instruction) Go to beginning bus interface I/O Bridge main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 4

Typical Computer Architecture CPU chip PC (Program Counter) register file ALU Main Components CPU Main Memory Input/Output Devices Buses bus interface System bus I/O Bridge Memory bus main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 5

Bus Bus: Electrical conduit carrying bits back and forth Serial bus: One bit after another Serial port, RS232, USB (Universal Serial Bus) Parallel bus: X bits in parallel Faster, but requires more lines (line per bit) Older printers, hardware buses in your computer For simplicity: X = word size i.e., buses carry W bits in parallel (W: Word size in bits) Possible to have multi-word buses, etc. Buses often have control signals e.g., letting receiver know when to expect bits e.g., setting intended recipient A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 6

Typical Bus Structure Connecting CPU and Memory A bus is a collection of parallel wires that carry address, data, and control signals. Buses are typically shared by multiple devices. CPU chip register file ALU system bus memory bus bus interface I/O bridge main memory A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 7

Memory Read Transaction (1) CPU places address A on the memory bus. register file Load operation: movl A, %eax %eax ALU bus interface I/O bridge A main memory 0 x A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 8

Memory Read Transaction (2) Main memory reads A from the memory bus, retrieves word x, and places it on the bus. register file Load operation: movl A, %eax %eax ALU I/O bridge main memory x 0 bus interface x A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 9

Memory Read Transaction (3) CPU read word x from the bus and copies it into register %eax. register file Load operation: movl A, %eax %eax x ALU I/O bridge main memory 0 bus interface x A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 10

Memory Write Transaction (1) CPU places address A on bus. Main memory reads it and waits for the corresponding data word to arrive. register file Store operation: movl %eax, A %eax y ALU bus interface I/O bridge A main memory 0 A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 11

Memory Write Transaction (2) CPU places data word y on the bus. register file Store operation: movl %eax, A %eax y ALU bus interface I/O bridge y main memory 0 A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 12

Memory Write Transaction (3) Main memory read data word y from the bus and stores it at address A. register file Store operation: movl %eax, A %eax y ALU I/O bridge main memory 0 bus interface y A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 13

There are many types of memories Volatile: Lose contents when power turned off RAM Non-volatile: Maintain memory after power turned off BIOS, ROM Firmware, Flash memory, EEPROM,... Hard disk drive (HDD), solid-state drive (SSD) Magnetic tapes, CD-Roms, CD-RW, DVD-RW,... Memories vary in access speed, cost In general, faster == more expensive (per byte) A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 15

Conventional DRAM Organization d x w DRAM: dw total bits organized as d supercells of size w bits 16 x 8 DRAM chip cols 0 1 2 3 (to CPU) memory controller 2 bits / addr 8 bits / data rows 0 1 2 3 supercell (2,1) internal row buffer A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 16

Random-Access Memory (RAM) Key features RAM is packaged as a chip. Basic storage unit is a cell (one bit per cell). Multiple RAM chips form a memory. Static RAM (SRAM) Retains value indefinitely, as long as it is kept powered. Relatively insensitive to disturbances such as electrical noise. Faster and more expensive than DRAM. Dynamic RAM (DRAM) Each cell stores bit with a capacitor and transistor. Value must be refreshed every 10-100 ns. Sensitive to disturbances. Slower and cheaper than SRAM. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 17

SRAM vs DRAM Summary Transistors. Access Need Need per bit time Refresh? Error Det.? Cost Applications SRAM 4 or 6 1X No Maybe 100x cache memories DRAM 1 10X Yes Yes 1X Main memories, frame buffers A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 19

Nonvolatile Memories DRAM and SRAM are volatile memories Lose information if powered off. Nonvolatile memories retain value even if powered off. Read-only memory (ROM): programmed during production Programmable ROM (PROM): can be programmed once Eraseable PROM (EPROM): can be bulk erased (UV, X-Ray) Electrically eraseable PROM (EEPROM): electronic erase capability Flash memory: EEPROMs. with partial (block-level) erase capability Wears out after about 100,000 erasings Uses: Firmware: programs stored in ROM (BIOS, disk/net controller) Solid-state disks (SSD) Disk caches A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 28

Input/Output Devices Connections to external world Each device connected to bus by either controller or adapter Controller: chips in device or on motherboard Adapter: cards that plug into slot on motherboard Devices often have their own processing capabilities Do some processing on-board Alert CPU when ready (via interrupt) This parallelizes processing A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 30

What s Inside A Disk Drive? Arm Spindle Platters Actuator BUS connector Electronics (including a processor and memory!) Image courtesy of Seagate Technology A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 31

Disk Geometry Disks consist of platters, each with two surfaces. Each surface consists of concentric rings called tracks. Each track consists of sectors separated by gaps. tracks surface track k gaps spindle sectors A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 32

Disk Geometry (Muliple-Platter View) Aligned tracks form a cylinder. cylinder k surface 0 surface 1 surface 2 surface 3 surface 4 surface 5 platter 0 platter 1 platter 2 spindle A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 33

Disk Capacity Capacity: maximum number of bits that can be stored. Vendors express capacity in units of gigabytes (GB), where 1 GB = 10^9. Capacity is determined by these technology factors: Recording density (bits/in): number of bits that can be squeezed into a 1 inch segment of a track. Track density (tracks/in): number of tracks that can be squeezed into a 1 inch radial segment. Areal density (bits/in2): product of recording and track density. Modern disks partition tracks into disjoint subsets called recording zones Each track in a zone has the same number of sectors, determined by the circumference of innermost track. Each zone has a different number of sectors/track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 34

Computing Disk Capacity Capacity = (# bytes/sector) x (avg. # sectors/track) x (# tracks/surface) x (# surfaces/platter) x (# platters/disk) Example: 512 bytes/sector 300 sectors/track (on average) 20,000 tracks/surface 2 surfaces/platter 5 platters/disk Capacity = 512 x 300 x 20000 x 2 x 5 = 30,720,000,000 = 30.72 GB A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 35

spindle Disk Operation (Single-Platter View) The disk surface spins at a fixed rotational rate The read/write head is attached to the end of the arm and flies over the disk surface on a thin cushion of air. spindle spindle spindle By moving radially, the arm can position the read/write head over any track. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 36

Disk Operation (Multi-Platter View) read/write heads move in unison from cylinder to cylinder arm spindle A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 37

Disk Structure - top view of single platter Surface organized into tracks Tracks divided into sectors A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 38

Disk Access Head in position above a track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 39

Disk Access Head in position above a track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 40

Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 41

Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 42

Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 43

Disk Access Seek After BLUE read Seek for RED Seek to red s track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 44

Disk Access Rotational Latency After BLUE read Seek for RED Rotational latency Wait for red sector to rotate around A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 45

Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 46

Disk Access Service Time Components After BLUE read Seek for RED Rotational latency After RED read Data transfer Seek Rotational latency Data transfer A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 47

Disk Access Time Average time to access some target sector approximated by : Taccess = Tavg seek + Tavg rotation + Tavg transfer Seek time (Tavg seek) Time to position heads over cylinder containing target sector. Typical Tavg seek = 3-9 ms Rotational latency (Tavg rotation) Time waiting for first bit of target sector to pass under r/w head. Tavg rotation = 1/2 x 1/RPMs x 60 sec/1 min Typical Tavg = 5600, 7200 rpm Transfer time (Tavg transfer) Time to read the bits in the target sector. Tavg transfer = 1/RPM x 1/(avg # sectors/track) x 60 secs/1 min. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 48

Disk Access Time Example Given: Rotational rate = 7,200 RPM Average seek time = 9 ms. Avg # sectors/track = 400. Derived: Tavg rotation = 1/2 x (60 secs/7200 RPM) x 1000 ms/sec = 4 ms. Tavg transfer = 60/7200 RPM x 1/400 secs/track x 1000 ms/sec = 0.02 ms Taccess = 9 ms + 4 ms + 0.02 ms Important points: Access time dominated by seek time and rotational latency. First bit in a sector is the most expensive, the rest are free. SRAM access time is about 4 ns/doubleword, DRAM about 60 ns Disk is about 40,000 times slower than SRAM, 2,500 times slower then DRAM. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 49

Logical Disk Blocks Modern disks present a simpler abstract view of the complex sector geometry: The set of available sectors is modeled as a sequence of b-sized logical blocks (0, 1, 2,...) Mapping between logical blocks and actual (physical) sectors Maintained by hardware/firmware device called disk controller. Converts requests for logical blocks into (surface,track,sector) triples. Allows controller to set aside spare cylinders for each zone. Accounts for the difference in formatted capacity and maximum capacity. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 50

I/O Bus CPU chip register file ALU system bus memory bus bus interface I/O bridge main memory USB controller graphics adapter I/O bus disk controller Expansion slots for other devices such as network adapters. mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 51

Reading a Disk Sector (1) CPU chip register file ALU CPU initiates a disk read by writing a command, logical block number, and destination memory address to a port (address) associated with disk controller. bus interface main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 52

Reading a Disk Sector (2) CPU chip register file ALU Disk controller reads the sector and performs a direct memory access (DMA) transfer into main memory. bus interface main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 53

Reading a Disk Sector (3) CPU chip register file ALU When the DMA transfer completes, the disk controller notifies the CPU with an interrupt (i.e., asserts a special interrupt pin on the CPU) bus interface main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 54

Solid State Disks (SSDs) I/O bus Solid State Disk (SSD) Requests to read and write logical disk blocks Flash translation layer Flash memory Block 0 Page 0 Page 1 Page P-1 Block B-1 Page 0 Page 1 Page P-1 Pages: 512KB to 4KB, Blocks: 32 to 128 pages Data read/written in units of pages. Page can be written only after its block has been erased A block wears out after about 100,000 repeated writes. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 55

SSD Performance Characteristics Sequential read tput 550 MB/s Sequential write tput 470 MB/s Random read tput 365 MB/s Random write tput 303 MB/s Avg seq read time 50 us Avg seq write time 60 us Sequential access faster than random access Common theme in the memory hierarchy Random writes are somewhat slower Erasing a block takes a long time (~1 ms) Modifying a block page requires all other pages to be copied to new block In earlier SSDs, the read/write gap was much larger. Source: Intel SSD 730 product specification. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 56

SSD Tradeoffs vs Rotating Disks Advantages No moving parts, faster, less power, more rugged Disadvantages Have the potential to wear out Mitigated by wear leveling logic in flash translation layer E.g. Intel SSD 730 guarantees 128 petabyte (128 x 10 15 bytes) of writes before they wear out In 2015, about 30 times more expensive per byte Applications MP3 players, smart phones, laptops Slowly appearing in desktops and servers A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 57

Executing Hello, World User types./hello in command-line shell Shell executes a series of instructions, that ultimately do: Identify a file called hello on disk a sequence of machine code bytes Load contents of file to main memory Each byte read and stored in some memory location In old machines, CPU had to be involved in this Now, use DMA to do this without involving CPU Address of first instruction of file loaded into PC (%rip) CPU begins executing code from this instruction To display, CPU sends data to monitor I/O device When done, OS loads address of own instructions into PC A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 58

?שאלות A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 59

The CPU-Memory Gap The gap widens between DRAM, disk, and CPU speeds. 100,000,000.0 10,000,000.0 Disk 1,000,000.0 100,000.0 SSD Time (ns) 10,000.0 1,000.0 100.0 10.0 DRAM Disk seek time SSD access time DRAM access time SRAM access time CPU cycle time Effective CPU cycle time 1.0 0.1 CPU 0.0 1985.0 1990.0 1995.0 2000.0 2003.0 2005.0 2010.0 2015.0 Year A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 60

Storage Trends SRAM Metric 1985 1990 1995 2000 2005 2010 2015 2015:1985 $/MB 2,900 320 256 100 75 60 25 116 access (ns) 150 35 15 3 2 1.5 1.3 115 DRAM Metric 1985 1990 1995 2000 2005 2010 2015 2015:1985 $/MB 880 100 30 1 0.1 0.06 0.02 44,000 access (ns) 200 100 70 60 50 40 20 10 typical size (MB) 0.256 4 16 64 2,000 8,000 16.000 62,500 Disk Metric 1985 1990 1995 2000 2005 2010 2015 2015:1985 $/GB 100,000 8,000 300 10 5 0.3 0.03 3,333,333 access (ms) 75 28 10 8 5 3 3 25 typical size (GB) 0.01 0.16 1 20 160 1,500 3,000 300,000 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 61

Key Question: How to have fast, cheap memory? A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 62

Principle of Locality Programs tend to reuse data and instructions: near those they have used recently Or same as those they have used recently Temporal locality: Recently referenced items are likely to be referenced in the near future. Spatial locality: Items with nearby addresses tend to be referenced close together in time. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 63

Example sum = 0; for (i = 0; i < n; i++) sum += a[i]; return sum; Locality Example: Data Reference array elements in succession (stride 1 reference pattern): Reference sum each iteration: Temporal locality Instructions Reference instructions in sequence: Spatial locality Cycle through loop repeatedly: Temporal locality Spatial locality A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 64

Memory Hierarchies Some fundamental and enduring properties of hardware and software: Fast storage technologies cost more per byte and have less capacity. The gap between CPU and main memory speed is widening. Well-written programs tend to exhibit good locality. These properties complement each other beautifully. They suggest an approach for organizing memory and storage systems known as a memory hierarchy. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 67

Example Memory Hierarchy Smaller, faster, and costlier (per byte) storage devices Larger, slower, and cheaper (per byte) storage devices L5: L6: L4: L3: L2: L1: L0: Regs L1 cache (SRAM) L2 cache (SRAM) L3 cache (SRAM) Main memory (DRAM) Local secondary storage (local disks) Remote secondary storage (e.g., Web servers) CPU registers hold words retrieved from the L1 cache. L1 cache holds cache lines retrieved from the L2 cache. L2 cache holds cache lines retrieved from L3 cache L3 cache holds cache lines retrieved from main memory. Main memory holds disk blocks retrieved from local disks. Local disks hold files retrieved from disks on remote servers A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 68

Caches Cache: A smaller, faster storage that acts as a staging area for a subset of the data in a larger, slower device. Fundamental idea of a memory hierarchy: For each k, the faster, smaller device at level k serves as a cache for the larger, slower device at level k+1. Why do memory hierarchies work? Programs tend to access the data at level k more often than they access the data at level k+1. Thus, the storage at level k+1 can be slower, and thus larger and cheaper per bit. Net effect: A large pool of memory that costs as much as the cheap storage near the bottom, but that serves data to programs at the rate of the fast storage near the top. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 69

Caching in a Memory Hierarchy Level k: 48 9 10 14 3 Smaller, faster, more expensive device at level k caches a subset of the blocks from level k+1 10 4 Data is copied between levels in block sized transfer units Level k+1: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Larger, slower, cheaper storage device at level k+1 is partitioned into blocks. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 70

General Caching Concepts Level k+1: Level k: 14 12 Request 14 12 0 1 2 3 12 4* 9 14 3 12 4* Request 12 0 1 2 3 4* 4 5 6 7 8 9 10 11 12 13 14 15 Program needs object d, which is stored in some block b. Cache hit Program finds b in the cache at level k. E.g., block 14. Cache miss b is not at level k, so level k cache must fetch it from level k+1. E.g., block 12. If level k cache is full, then some current block must be replaced (evicted). Which one is the victim? Placement policy: where can the new block go? E.g., b mod 4 Replacement policy: which block should be evicted? E.g., LRU A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 71

General Caching Concepts Types of cache misses: Cold (compulsary) miss Cold misses occur because the cache is empty. Conflict miss Most caches limit blocks at level k+1 to a small subset (sometimes a singleton) of the block positions at level k. e.g. Block i at level k+1 is placed in block (i mod 4) at level k+1. Conflict misses occur when the level k cache is large enough, but multiple data objects all map to the same level k block. E.g. Referencing blocks 0, 8, 0, 8, 0, 8,... would miss every time. Capacity miss Occurs when the set of active cache blocks (working set) is larger than the cache. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 72

Cache Memories Cache memories are small, fast SRAM-based memories managed automatically in hardware. Hold frequently accessed blocks of main memory CPU looks first for data in L1, then in L2, then in main memory. Typical bus structure: CPU chip register file L1 ALU cache cache bus system bus memory bus L2 cache bus interface I/O bridge main memory A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 73

?שאלות A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 74

General Organization of a Cache Memory Cache is an array of sets. 1 valid bit per line t tag bits per line B = 2 b bytes per cache block Each set contains one or more lines. Each line holds a block of data. set 0: valid valid valid tag tag tag 0 0 0 1 1 1 B 1 B 1 B 1 E lines per set S = 2 s sets set 1: valid tag 0 1 B 1 valid tag 0 1 B 1 set S 1: valid tag 0 1 B 1 Cache size: C = B x E x S data bytes A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 75

Addressing Caches Address A: t bits s bits b bits set 0: v v tag tag 0 0 1 1 B 1 B 1 m 1 <tag> <set index> <block offset> 0 set 1: set S 1: v v v v tag tag tag tag 0 0 0 0 1 1 1 1 B 1 B 1 B 1 B 1 The word at address A is in the cache if the tag bits in one of the <valid> lines in set <set index> match <tag>. The word contents begin at offset <block offset> bytes from the beginning of the block. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 76

Direct-Mapped Cache Simplest kind of cache Characterized by exactly one line per set (E=1). set 0: valid tag cache block E=1 lines per set set 1: valid tag cache block set S 1: valid tag cache block A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 77

Accessing Direct-Mapped Caches Set selection Use the set index bits to determine the set of interest. set 0: valid tag cache block selected set set 1: valid tag cache block t bits s bits 0 0 0 0 1 b bits set S 1: valid tag cache block m 1 tag set index block offset 0 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 78

Accessing Direct-Mapped Caches Line matching and word selection Line matching: Find a valid line in the selected set with a matching tag Word selection: Then extract the word (here 32bits made from 4 bytes B 0..B 3 ) =1? (1) The valid bit must be set selected set (i): 1 0 1 2 3 4 5 6 7 0110 B 0 B 1 B 2 B 3 (2) The tag bits in the cache line must match the tag bits in the address m 1 =? t bits s bits b bits 0110 i 100 tag set index block offset 0 (3) If (1) and (2), then cache hit, and block offset selects starting byte. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 79

Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [0000 2 ], 1 [0001 2 ], 13 [1101 2 ], 8 [1000 2 ], 0 [0000 2 ] (1) 0 [0000 2 ] (cold miss) v tag data 1 0 M[0] m[1] M[1] m[0] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 81

Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [0000 2 ], 1 [0001 2 ], 13 [1101 2 ], 8 [1000 2 ], 0 [0000 2 ] (1) v tag data 1 0 M[0] m[1] M[1] m[0] (3) 13 [1101 2 ] (cold miss) v tag data 1 0 m[1] M[0] m[0] M[1] 1 1 m[13] M[12] m[12] M[13] (4) 8 [1000 2 ] (conflict miss) v tag data 1 1 m[9] M[8] m[8] M[9] 1 1 M[12] M[13] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 84

Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [0000 2 ], 1 [0001 2 ], 13 [1101 2 ], 8 [1000 2 ], 0 [0000 2 ] (1) v tag data 1 0 M[0] m[1] M[1] m[0] (3) 13 [1101 2 ] (miss) v tag data 1 0 m[1] M[0] m[0] M[1] 1 1 m[13] M[12] m[12] M[13] (4) 8 [1000 2 ] (conflict miss) v tag data 1 1 M[8] m[9] M[9] m[8] 0 [0000 2 ] (conflict miss) v tag data 1 00 m[1] M[0] m[0] M[1] 1 1 M[12] M[13] 1 11 m[13] M[12] M[13] m[12] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 85 (5)

Why Use Middle Bits as Index? 00 01 10 11 4 line Cache High-Order Bit Indexing Adjacent memory lines would map to same cache entry Poor use of spatial locality Middle-Order Bit Indexing Consecutive memory lines map to different cache lines Can hold C-byte region of address space in cache at one time 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 High Order Bit Indexing 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Middle Order Bit Indexing A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 86

Set Associative Caches Characterized by more than one line per set set 0: valid tag cache block valid tag cache block E=2 lines per set set 1: set S 1: valid tag cache block valid tag cache block valid tag cache block valid tag cache block A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 87

Accessing Set Associative Caches Set selection identical to direct-mapped cache set 0: valid valid tag tag cache block cache block Selected set set 1: valid valid tag tag cache block cache block t bits s bits 0 0 0 0 1 b bits set S 1: valid valid tag tag cache block cache block m 1 tag set index block offset 0 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 88

Accessing Set Associative Caches Line matching and word selection must compare the tag in each valid line in the selected set. =1? (1) The valid bit must be set. 0 1 2 3 4 5 6 7 selected set (i): 1 1001 1 0110 b 0 b 1 b 2 b 3 (2) The tag bits in one of the cache lines must match the tag bits in the address m 1 =? t bits 0110 s bits i b bits 100 tag set index block offset (3) If (1) and (2), then cache hit, and block offset selects starting byte. The four bytes here make up a full 32bit word 0 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 89

What about writes? Multiple copies of data exist: L1, L2, L3, Main Memory, Disk What to do on a write-hit? Write-through (write immediately to one level down) Write-back (defer write to one level down until replacement of line) Need a dirty bit (line different from memory or not) What to do on a write-miss? Write-allocate (load into cache, update line in cache) Good if more writes to the location follow No-write-allocate (writes straight to one level down, does not load into cache) Typical Write-through + No-write-allocate Write-back + Write-allocate A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 90

Intel Core i7 Cache Hierarchy Processor package Core 0 Regs Core 3 Regs L1 i-cache and d-cache: 32 KB, 8-way, Access: 4 cycles L1 d-cache L1 i-cache L1 d-cache L1 i-cache L2 unified cache: 256 KB, 8-way, Access: 10 cycles L2 unified cache L2 unified cache L3 unified cache: 8 MB, 16-way, Access: 40-75 cycles L3 unified cache (shared by all cores) Block size: 64 bytes for all caches. Main memory A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 91

Intel Pentium Cache Hierarchy Regs. L1 Data 1 cycle latency 16 KB 4 way assoc Write through 32B lines L1 Instruction 16 KB, 4 way 32B lines L2 L2 Unified Unified 128KB 2 128KB 2 MB MB 4 way 4 way assoc assoc Write back Write back Write Write allocate allocate 32B 32B lines lines Main Main Memory Memory Up Up to to 4GB 4GB Processor Processor Chip Chip A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 92

?שאלות A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 93