Computer Organization: A Programmer's Perspective

Transcription

1 A Programmer's Perspective Computer Architecture and The Memory Hierarchy Gal A. Kaminka

2 Typical Computer Architecture CPU chip PC (Program Counter) register file ALU Main Components CPU Main Memory Input/Output Devices Buses bus interface I/O Bridge main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 2

3 Typical Computer Architecture CPU chip PC (Program Counter) register file ALU Main Components CPU Main Memory Input/Output Devices Buses bus interface I/O Bridge main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 3

4 Typical Computer Architecture CPU chip PC (Program Counter) register file ALU CPU Algorithm: Read instruction from address in PC Interpret bits in instruction Perform instruction using ALU Update PC (sometimes based on instruction) Go to beginning bus interface I/O Bridge main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 4

5 Typical Computer Architecture CPU chip PC (Program Counter) register file ALU Main Components CPU Main Memory Input/Output Devices Buses bus interface System bus I/O Bridge Memory bus main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 5

6 Bus Bus: Electrical conduit carrying bits back and forth Serial bus: One bit after another Serial port, RS232, USB (Universal Serial Bus) Parallel bus: X bits in parallel Faster, but requires more lines (line per bit) Older printers, hardware buses in your computer For simplicity: X = word size i.e., buses carry W bits in parallel (W: Word size in bits) Possible to have multi-word buses, etc. Buses often have control signals e.g., letting receiver know when to expect bits e.g., setting intended recipient A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 6

7 Typical Bus Structure Connecting CPU and Memory A bus is a collection of parallel wires that carry address, data, and control signals. Buses are typically shared by multiple devices. CPU chip register file ALU system bus memory bus bus interface I/O bridge main memory A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 7

8 Memory Read Transaction (1) CPU places address A on the memory bus. register file Load operation: movl A, %eax %eax ALU bus interface I/O bridge A main memory 0 x A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 8

9 Memory Read Transaction (2) Main memory reads A from the memory bus, retrieves word x, and places it on the bus. register file Load operation: movl A, %eax %eax ALU I/O bridge main memory x 0 bus interface x A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 9

10 Memory Read Transaction (3) CPU read word x from the bus and copies it into register %eax. register file Load operation: movl A, %eax %eax x ALU I/O bridge main memory 0 bus interface x A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 10

11 Memory Write Transaction (1) CPU places address A on bus. Main memory reads it and waits for the corresponding data word to arrive. register file Store operation: movl %eax, A %eax y ALU bus interface I/O bridge A main memory 0 A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 11

12 Memory Write Transaction (2) CPU places data word y on the bus. register file Store operation: movl %eax, A %eax y ALU bus interface I/O bridge y main memory 0 A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 12

13 Memory Write Transaction (3) Main memory read data word y from the bus and stores it at address A. register file Store operation: movl %eax, A %eax y ALU I/O bridge main memory 0 bus interface y A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 13

14 Typical Computer Architecture CPU chip PC (Program Counter) register file ALU Main Components CPU Main Memory Input/Output Devices Buses bus interface I/O Bridge main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 14

15 There are many types of memories Volatile: Lose contents when power turned off RAM Non-volatile: Maintain memory after power turned off BIOS, ROM Firmware, Flash memory, EEPROM,... Hard disk drive (HDD), solid-state drive (SSD) Magnetic tapes, CD-Roms, CD-RW, DVD-RW,... Memories vary in access speed, cost In general, faster == more expensive (per byte) A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 15

16 Conventional DRAM Organization d x w DRAM: dw total bits organized as d supercells of size w bits 16 x 8 DRAM chip cols (to CPU) memory controller 2 bits / addr 8 bits / data rows supercell (2,1) internal row buffer A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 16

17 Random-Access Memory (RAM) Key features RAM is packaged as a chip. Basic storage unit is a cell (one bit per cell). Multiple RAM chips form a memory. Static RAM (SRAM) Retains value indefinitely, as long as it is kept powered. Relatively insensitive to disturbances such as electrical noise. Faster and more expensive than DRAM. Dynamic RAM (DRAM) Each cell stores bit with a capacitor and transistor. Value must be refreshed every ns. Sensitive to disturbances. Slower and cheaper than SRAM. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 17

18 Memory Write Transaction (2) CPU places data word y on the bus. register file Store operation: movl %eax, A %eax y ALU bus interface I/O bridge y main memory 0 A A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 18

19 SRAM vs DRAM Summary Transistors. Access Need Need per bit time Refresh? Error Det.? Cost Applications SRAM 4 or 6 1X No Maybe 100x cache memories DRAM 1 10X Yes Yes 1X Main memories, frame buffers A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 19

20 There are many types of memories Volatile: Lose contents when power turned off RAM Non-volatile: Maintain memory after power turned off BIOS, ROM Firmware, Flash memory, EEPROM,... Hard disk drive (HDD), solid-state drive (SSD) Magnetic tapes, CD-Roms, CD-RW, DVD-RW,... Memories vary in access speed, cost In general, faster == more expensive (per byte) A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 27

21 Nonvolatile Memories DRAM and SRAM are volatile memories Lose information if powered off. Nonvolatile memories retain value even if powered off. Read-only memory (ROM): programmed during production Programmable ROM (PROM): can be programmed once Eraseable PROM (EPROM): can be bulk erased (UV, X-Ray) Electrically eraseable PROM (EEPROM): electronic erase capability Flash memory: EEPROMs. with partial (block-level) erase capability Wears out after about 100,000 erasings Uses: Firmware: programs stored in ROM (BIOS, disk/net controller) Solid-state disks (SSD) Disk caches A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 28

22 Typical Computer Architecture CPU chip PC (Program Counter) register file ALU Main Components CPU Main Memory Input/Output Devices Buses bus interface I/O Bridge main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 29

23 Input/Output Devices Connections to external world Each device connected to bus by either controller or adapter Controller: chips in device or on motherboard Adapter: cards that plug into slot on motherboard Devices often have their own processing capabilities Do some processing on-board Alert CPU when ready (via interrupt) This parallelizes processing A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 30

24 What s Inside A Disk Drive? Arm Spindle Platters Actuator BUS connector Electronics (including a processor and memory!) Image courtesy of Seagate Technology A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 31

25 Disk Geometry Disks consist of platters, each with two surfaces. Each surface consists of concentric rings called tracks. Each track consists of sectors separated by gaps. tracks surface track k gaps spindle sectors A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 32

26 Disk Geometry (Muliple-Platter View) Aligned tracks form a cylinder. cylinder k surface 0 surface 1 surface 2 surface 3 surface 4 surface 5 platter 0 platter 1 platter 2 spindle A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 33

27 Disk Capacity Capacity: maximum number of bits that can be stored. Vendors express capacity in units of gigabytes (GB), where 1 GB = 10^9. Capacity is determined by these technology factors: Recording density (bits/in): number of bits that can be squeezed into a 1 inch segment of a track. Track density (tracks/in): number of tracks that can be squeezed into a 1 inch radial segment. Areal density (bits/in2): product of recording and track density. Modern disks partition tracks into disjoint subsets called recording zones Each track in a zone has the same number of sectors, determined by the circumference of innermost track. Each zone has a different number of sectors/track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 34

28 Computing Disk Capacity Capacity = (# bytes/sector) x (avg. # sectors/track) x (# tracks/surface) x (# surfaces/platter) x (# platters/disk) Example: 512 bytes/sector 300 sectors/track (on average) 20,000 tracks/surface 2 surfaces/platter 5 platters/disk Capacity = 512 x 300 x x 2 x 5 = 30,720,000,000 = GB A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 35

29 spindle Disk Operation (Single-Platter View) The disk surface spins at a fixed rotational rate The read/write head is attached to the end of the arm and flies over the disk surface on a thin cushion of air. spindle spindle spindle By moving radially, the arm can position the read/write head over any track. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 36

30 Disk Operation (Multi-Platter View) read/write heads move in unison from cylinder to cylinder arm spindle A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 37

31 Disk Structure - top view of single platter Surface organized into tracks Tracks divided into sectors A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 38

32 Disk Access Head in position above a track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 39

33 Disk Access Head in position above a track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 40

34 Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 41

35 Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 42

36 Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 43

37 Disk Access Seek After BLUE read Seek for RED Seek to red s track A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 44

38 Disk Access Rotational Latency After BLUE read Seek for RED Rotational latency Wait for red sector to rotate around A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 45

39 Disk Access Read About to read blue sector A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 46

40 Disk Access Service Time Components After BLUE read Seek for RED Rotational latency After RED read Data transfer Seek Rotational latency Data transfer A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 47

41 Disk Access Time Average time to access some target sector approximated by : Taccess = Tavg seek + Tavg rotation + Tavg transfer Seek time (Tavg seek) Time to position heads over cylinder containing target sector. Typical Tavg seek = 3-9 ms Rotational latency (Tavg rotation) Time waiting for first bit of target sector to pass under r/w head. Tavg rotation = 1/2 x 1/RPMs x 60 sec/1 min Typical Tavg = 5600, 7200 rpm Transfer time (Tavg transfer) Time to read the bits in the target sector. Tavg transfer = 1/RPM x 1/(avg # sectors/track) x 60 secs/1 min. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 48

42 Disk Access Time Example Given: Rotational rate = 7,200 RPM Average seek time = 9 ms. Avg # sectors/track = 400. Derived: Tavg rotation = 1/2 x (60 secs/7200 RPM) x 1000 ms/sec = 4 ms. Tavg transfer = 60/7200 RPM x 1/400 secs/track x 1000 ms/sec = 0.02 ms Taccess = 9 ms + 4 ms ms Important points: Access time dominated by seek time and rotational latency. First bit in a sector is the most expensive, the rest are free. SRAM access time is about 4 ns/doubleword, DRAM about 60 ns Disk is about 40,000 times slower than SRAM, 2,500 times slower then DRAM. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 49

43 Logical Disk Blocks Modern disks present a simpler abstract view of the complex sector geometry: The set of available sectors is modeled as a sequence of b-sized logical blocks (0, 1, 2,...) Mapping between logical blocks and actual (physical) sectors Maintained by hardware/firmware device called disk controller. Converts requests for logical blocks into (surface,track,sector) triples. Allows controller to set aside spare cylinders for each zone. Accounts for the difference in formatted capacity and maximum capacity. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 50

44 I/O Bus CPU chip register file ALU system bus memory bus bus interface I/O bridge main memory USB controller graphics adapter I/O bus disk controller Expansion slots for other devices such as network adapters. mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 51

45 Reading a Disk Sector (1) CPU chip register file ALU CPU initiates a disk read by writing a command, logical block number, and destination memory address to a port (address) associated with disk controller. bus interface main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 52

46 Reading a Disk Sector (2) CPU chip register file ALU Disk controller reads the sector and performs a direct memory access (DMA) transfer into main memory. bus interface main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 53

47 Reading a Disk Sector (3) CPU chip register file ALU When the DMA transfer completes, the disk controller notifies the CPU with an interrupt (i.e., asserts a special interrupt pin on the CPU) bus interface main memory I/O bus USB controller graphics adapter disk controller mouse keyboard monitor disk A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 54

48 Solid State Disks (SSDs) I/O bus Solid State Disk (SSD) Requests to read and write logical disk blocks Flash translation layer Flash memory Block 0 Page 0 Page 1 Page P-1 Block B-1 Page 0 Page 1 Page P-1 Pages: 512KB to 4KB, Blocks: 32 to 128 pages Data read/written in units of pages. Page can be written only after its block has been erased A block wears out after about 100,000 repeated writes. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 55

49 SSD Performance Characteristics Sequential read tput 550 MB/s Sequential write tput 470 MB/s Random read tput 365 MB/s Random write tput 303 MB/s Avg seq read time 50 us Avg seq write time 60 us Sequential access faster than random access Common theme in the memory hierarchy Random writes are somewhat slower Erasing a block takes a long time (~1 ms) Modifying a block page requires all other pages to be copied to new block In earlier SSDs, the read/write gap was much larger. Source: Intel SSD 730 product specification. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 56

50 SSD Tradeoffs vs Rotating Disks Advantages No moving parts, faster, less power, more rugged Disadvantages Have the potential to wear out Mitigated by wear leveling logic in flash translation layer E.g. Intel SSD 730 guarantees 128 petabyte (128 x bytes) of writes before they wear out In 2015, about 30 times more expensive per byte Applications MP3 players, smart phones, laptops Slowly appearing in desktops and servers A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 57

51 Executing Hello, World User types./hello in command-line shell Shell executes a series of instructions, that ultimately do: Identify a file called hello on disk a sequence of machine code bytes Load contents of file to main memory Each byte read and stored in some memory location In old machines, CPU had to be involved in this Now, use DMA to do this without involving CPU Address of first instruction of file loaded into PC (%rip) CPU begins executing code from this instruction To display, CPU sends data to monitor I/O device When done, OS loads address of own instructions into PC A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 58

52 ?שאלות A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 59

53 The CPU-Memory Gap The gap widens between DRAM, disk, and CPU speeds. 100,000, ,000,000.0 Disk 1,000, ,000.0 SSD Time (ns) 10, , DRAM Disk seek time SSD access time DRAM access time SRAM access time CPU cycle time Effective CPU cycle time CPU Year A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 60

54 Storage Trends SRAM Metric :1985 $/MB 2, access (ns) DRAM Metric :1985 $/MB ,000 access (ns) typical size (MB) ,000 8, ,500 Disk Metric :1985 $/GB 100,000 8, ,333,333 access (ms) typical size (GB) ,500 3, ,000 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 61

55 Key Question: How to have fast, cheap memory? A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 62

56 Principle of Locality Programs tend to reuse data and instructions: near those they have used recently Or same as those they have used recently Temporal locality: Recently referenced items are likely to be referenced in the near future. Spatial locality: Items with nearby addresses tend to be referenced close together in time. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 63

57 Example sum = 0; for (i = 0; i < n; i++) sum += a[i]; return sum; Locality Example: Data Reference array elements in succession (stride 1 reference pattern): Reference sum each iteration: Temporal locality Instructions Reference instructions in sequence: Spatial locality Cycle through loop repeatedly: Temporal locality Spatial locality A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 64

58 Memory Hierarchies Some fundamental and enduring properties of hardware and software: Fast storage technologies cost more per byte and have less capacity. The gap between CPU and main memory speed is widening. Well-written programs tend to exhibit good locality. These properties complement each other beautifully. They suggest an approach for organizing memory and storage systems known as a memory hierarchy. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 67

59 Example Memory Hierarchy Smaller, faster, and costlier (per byte) storage devices Larger, slower, and cheaper (per byte) storage devices L5: L6: L4: L3: L2: L1: L0: Regs L1 cache (SRAM) L2 cache (SRAM) L3 cache (SRAM) Main memory (DRAM) Local secondary storage (local disks) Remote secondary storage (e.g., Web servers) CPU registers hold words retrieved from the L1 cache. L1 cache holds cache lines retrieved from the L2 cache. L2 cache holds cache lines retrieved from L3 cache L3 cache holds cache lines retrieved from main memory. Main memory holds disk blocks retrieved from local disks. Local disks hold files retrieved from disks on remote servers A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 68

60 Caches Cache: A smaller, faster storage that acts as a staging area for a subset of the data in a larger, slower device. Fundamental idea of a memory hierarchy: For each k, the faster, smaller device at level k serves as a cache for the larger, slower device at level k+1. Why do memory hierarchies work? Programs tend to access the data at level k more often than they access the data at level k+1. Thus, the storage at level k+1 can be slower, and thus larger and cheaper per bit. Net effect: A large pool of memory that costs as much as the cheap storage near the bottom, but that serves data to programs at the rate of the fast storage near the top. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 69

61 Caching in a Memory Hierarchy Level k: Smaller, faster, more expensive device at level k caches a subset of the blocks from level k Data is copied between levels in block sized transfer units Level k+1: Larger, slower, cheaper storage device at level k+1 is partitioned into blocks. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 70

62 General Caching Concepts Level k+1: Level k: Request * * Request * Program needs object d, which is stored in some block b. Cache hit Program finds b in the cache at level k. E.g., block 14. Cache miss b is not at level k, so level k cache must fetch it from level k+1. E.g., block 12. If level k cache is full, then some current block must be replaced (evicted). Which one is the victim? Placement policy: where can the new block go? E.g., b mod 4 Replacement policy: which block should be evicted? E.g., LRU A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 71

63 General Caching Concepts Types of cache misses: Cold (compulsary) miss Cold misses occur because the cache is empty. Conflict miss Most caches limit blocks at level k+1 to a small subset (sometimes a singleton) of the block positions at level k. e.g. Block i at level k+1 is placed in block (i mod 4) at level k+1. Conflict misses occur when the level k cache is large enough, but multiple data objects all map to the same level k block. E.g. Referencing blocks 0, 8, 0, 8, 0, 8,... would miss every time. Capacity miss Occurs when the set of active cache blocks (working set) is larger than the cache. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 72

64 Cache Memories Cache memories are small, fast SRAM-based memories managed automatically in hardware. Hold frequently accessed blocks of main memory CPU looks first for data in L1, then in L2, then in main memory. Typical bus structure: CPU chip register file L1 ALU cache cache bus system bus memory bus L2 cache bus interface I/O bridge main memory A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 73

65 ?שאלות A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 74

66 General Organization of a Cache Memory Cache is an array of sets. 1 valid bit per line t tag bits per line B = 2 b bytes per cache block Each set contains one or more lines. Each line holds a block of data. set 0: valid valid valid tag tag tag B 1 B 1 B 1 E lines per set S = 2 s sets set 1: valid tag 0 1 B 1 valid tag 0 1 B 1 set S 1: valid tag 0 1 B 1 Cache size: C = B x E x S data bytes A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 75

67 Addressing Caches Address A: t bits s bits b bits set 0: v v tag tag B 1 B 1 m 1 <tag> <set index> <block offset> 0 set 1: set S 1: v v v v tag tag tag tag B 1 B 1 B 1 B 1 The word at address A is in the cache if the tag bits in one of the <valid> lines in set <set index> match <tag>. The word contents begin at offset <block offset> bytes from the beginning of the block. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 76

68 Direct-Mapped Cache Simplest kind of cache Characterized by exactly one line per set (E=1). set 0: valid tag cache block E=1 lines per set set 1: valid tag cache block set S 1: valid tag cache block A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 77

69 Accessing Direct-Mapped Caches Set selection Use the set index bits to determine the set of interest. set 0: valid tag cache block selected set set 1: valid tag cache block t bits s bits b bits set S 1: valid tag cache block m 1 tag set index block offset 0 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 78

70 Accessing Direct-Mapped Caches Line matching and word selection Line matching: Find a valid line in the selected set with a matching tag Word selection: Then extract the word (here 32bits made from 4 bytes B 0..B 3 ) =1? (1) The valid bit must be set selected set (i): B 0 B 1 B 2 B 3 (2) The tag bits in the cache line must match the tag bits in the address m 1 =? t bits s bits b bits 0110 i 100 tag set index block offset 0 (3) If (1) and (2), then cache hit, and block offset selects starting byte. A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 79

71 Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [ ], 1 [ ], 13 [ ], 8 [ ], 0 [ ] (1) 0 [ ] (cold miss) v tag data 1 0 m[1] m[0] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 80

72 Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [ ], 1 [ ], 13 [ ], 8 [ ], 0 [ ] (1) 0 [ ] (cold miss) v tag data 1 0 M[0] m[1] M[1] m[0] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 81

73 Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [ ], 1 [ ], 13 [ ], 8 [ ], 0 [ ] (2) 1 [ ] (hit!) v tag data 1 0 M[0] m[1] M[1] m[0] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 82

74 Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [ ], 1 [ ], 13 [ ], 8 [ ], 0 [ ] (1) v tag data 1 0 M[0] m[1] M[1] m[0] (3) 13 [ ] (cold miss) v tag data 1 0 M[0] m[1] M[1] m[0] 1 1 M[12] m[13] M[13] m[12] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 83

75 Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [ ], 1 [ ], 13 [ ], 8 [ ], 0 [ ] (1) v tag data 1 0 M[0] m[1] M[1] m[0] (3) 13 [ ] (cold miss) v tag data 1 0 m[1] M[0] m[0] M[1] 1 1 m[13] M[12] m[12] M[13] (4) 8 [ ] (conflict miss) v tag data 1 1 m[9] M[8] m[8] M[9] 1 1 M[12] M[13] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 84

76 Direct-Mapped Cache Simulation t=1 s=2 b=1 x xx x M=16 byte addresses, B=2 bytes/block, S=4 sets, E=1 entry/set Address trace (reads): 0 [ ], 1 [ ], 13 [ ], 8 [ ], 0 [ ] (1) v tag data 1 0 M[0] m[1] M[1] m[0] (3) 13 [ ] (miss) v tag data 1 0 m[1] M[0] m[0] M[1] 1 1 m[13] M[12] m[12] M[13] (4) 8 [ ] (conflict miss) v tag data 1 1 M[8] m[9] M[9] m[8] 0 [ ] (conflict miss) v tag data 1 00 m[1] M[0] m[0] M[1] 1 1 M[12] M[13] 1 11 m[13] M[12] M[13] m[12] A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 85 (5)

77 Why Use Middle Bits as Index? line Cache High-Order Bit Indexing Adjacent memory lines would map to same cache entry Poor use of spatial locality Middle-Order Bit Indexing Consecutive memory lines map to different cache lines Can hold C-byte region of address space in cache at one time High Order Bit Indexing Middle Order Bit Indexing A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 86

78 Set Associative Caches Characterized by more than one line per set set 0: valid tag cache block valid tag cache block E=2 lines per set set 1: set S 1: valid tag cache block valid tag cache block valid tag cache block valid tag cache block A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 87

79 Accessing Set Associative Caches Set selection identical to direct-mapped cache set 0: valid valid tag tag cache block cache block Selected set set 1: valid valid tag tag cache block cache block t bits s bits b bits set S 1: valid valid tag tag cache block cache block m 1 tag set index block offset 0 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 88

80 Accessing Set Associative Caches Line matching and word selection must compare the tag in each valid line in the selected set. =1? (1) The valid bit must be set selected set (i): b 0 b 1 b 2 b 3 (2) The tag bits in one of the cache lines must match the tag bits in the address m 1 =? t bits 0110 s bits i b bits 100 tag set index block offset (3) If (1) and (2), then cache hit, and block offset selects starting byte. The four bytes here make up a full 32bit word 0 A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 89

81 What about writes? Multiple copies of data exist: L1, L2, L3, Main Memory, Disk What to do on a write-hit? Write-through (write immediately to one level down) Write-back (defer write to one level down until replacement of line) Need a dirty bit (line different from memory or not) What to do on a write-miss? Write-allocate (load into cache, update line in cache) Good if more writes to the location follow No-write-allocate (writes straight to one level down, does not load into cache) Typical Write-through + No-write-allocate Write-back + Write-allocate A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 90

82 Intel Core i7 Cache Hierarchy Processor package Core 0 Regs Core 3 Regs L1 i-cache and d-cache: 32 KB, 8-way, Access: 4 cycles L1 d-cache L1 i-cache L1 d-cache L1 i-cache L2 unified cache: 256 KB, 8-way, Access: 10 cycles L2 unified cache L2 unified cache L3 unified cache: 8 MB, 16-way, Access: cycles L3 unified cache (shared by all cores) Block size: 64 bytes for all caches. Main memory A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 91

83 Intel Pentium Cache Hierarchy Regs. L1 Data 1 cycle latency 16 KB 4 way assoc Write through 32B lines L1 Instruction 16 KB, 4 way 32B lines L2 L2 Unified Unified 128KB 2 128KB 2 MB MB 4 way 4 way assoc assoc Write back Write back Write Write allocate allocate 32B 32B lines lines Main Main Memory Memory Up Up to to 4GB 4GB Processor Processor Chip Chip A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 92

84 ?שאלות A Programmer's Perspective Based on class notes by Bryant and O'Hallaron 93