Chapter 3. SEQUENTIAL Logic Design. Digital Design and Computer Architecture, 2 nd Edition. David Money Harris and Sarah L. Harris.

SEQUENTIAL Logic Design Chapter 3 Digital Design and Computer Architecture, 2 nd Edition David Money Harris and Sarah L. Harris Chapter 3 <1>

Chapter 3 :: Topics Introduction Latches and Flip-Flops Synchronous Logic Design Finite State Machines Timing of Sequential Logic Parallelism Chapter 3 <2>

Introduction Outputs of sequential logic depend on current and prior input values it has memory. Some definitions: State: all the information about a circuit necessary to explain its future behavior Latches and flip-flops: state elements that store one bit of state Synchronous sequential circuits: combinational logic followed by a bank of flip-flops Chapter 3 <3>

Sequential Circuits Give sequence to events Have memory (short-term) Use feedback from output to input to store information Chapter 3 <4>

State Elements The state of a circuit influences its future behavior State elements store state Bistable circuit SR Latch D Latch D Flip-flop Chapter 3 <5>

Bistable Circuit Fundamental building block of other state elements Two outputs: Q, Q No inputs I2 Q I1 Q I1 Q I2 Q Chapter 3 <6>

Bistable Circuit Analysis Consider the two possible cases: Q = : then Q = 1, Q = (consistent) 1 I1 Q Q = 1: then Q =, Q = 1 (consistent) Stores 1 bit of state in the state variable, Q (or Q) 1 But there are no inputs to control the state I2 I2 I1 1 1 Q Q Q Chapter 3 <7>

SR (Set/Reset) Latch SR Latch R N1 Q Consider the four possible cases: S = 1, R = S =, R = 1 S =, R = S = 1, R = 1 S N2 Q Chapter 3 <8>

SR Latch Analysis S = 1, R = : then Q = 1 and Q = R N1 1 Q 1 / 1 S N2 Q S =, R = 1: then Q = 1 and Q = R 1 1 N1 Q S N2 1 Q Chapter 3 <9>

SR Latch Analysis S =, R = : then Q = Q prev R Q prev = Q prev = 1 R N1 Q N1 1 Q S = 1, R = 1: then Q =, Q = S N2 Q S N2 Q R 1 N1 Q S 1 N2 Q Chapter 3 <1>

SR Latch Analysis S =, R = : then Q = Q prev Memory! R Q prev = Q prev = 1 R N1 Q N1 Q S = 1, R = 1: then Q =, Q = Invalid State Q NOT Q S R S N2 1 1 N1 N2 Q Q Q S N2 Q Chapter 3 <11>

SR Latch Symbol SR stands for Set/Reset Latch Stores one bit of state (Q) Control what value is being with S, R inputs Set: Make the output 1 (S = 1, R =, Q = 1) Reset: Make the output (S =, R = 1, Q = ) Must do something to avoid invalid state (when S = R = 1) Chapter 3 <12> SR Latch Symbol R S Q Q

D Latch Two inputs: CLK, D CLK: controls when the output changes D (the data input): controls what the output changes to Function When CLK = 1, D passes through to Q (transparent) When CLK =, Q holds its previous value (opaque) Avoids invalid case when Q NOT Q D Latch Symbol D CLK Q Q Chapter 3 <13>

D Latch Internal Circuit CLK D R R Q Q CLK D S S Q Q D Q Q CLK D X 1 1 1 D S R Q Q Chapter 3 <14>

D Latch Internal Circuit CLK D R R Q Q CLK D S S Q Q D Q Q CLK D X 1 1 1 D X 1 S R Q Q Q prev 1 1 1 1 Q prev Chapter 3 <15>

D Flip-Flop Inputs: CLK, D Function Samples D on rising edge of CLK When CLK rises from to 1, D passes through to Q Otherwise, Q holds its previous value Q changes only on rising edge of CLK Called edge-triggered Activated on the clock edge D D Flip-Flop Symbols Q Q Chapter 3 <16>

D Flip-Flop Internal Circuit Two back-to-back latches (L1 and L2) controlled by complementary clocks When CLK = CLK L1 is transparent L2 is opaque CLK CLK D passes through to N1 N1 When CLK = 1 D D Q D Q Q L2 is transparent L1 is opaque N1 passes through to Q L1 Q L2 Q Q Thus, on the edge of the clock (when CLK rises from 1) D passes through to Q Chapter 3 <17>

D Latch vs. D Flip-Flop CLK D Q Q D Q Q CLK D Q (latch) Q (flop) Chapter 3 <18>

D Latch vs. D Flip-Flop CLK D Q Q D Q Q CLK D Q (latch) Q (flop) Chapter 3 <19>

Registers CLK D D Q Q CLK D 1 D Q Q 1 D 3: 4 4 Q 3: D 2 D Q Q 2 D 3 D Q Q 3 Chapter 3 <2>

Enabled Flip-Flops Inputs: CLK, D, EN The enable input (EN) controls when new data (D) is Function EN = 1: D passes through to Q on the clock edge EN = : the flip-flop retains its previous state EN Internal Circuit CLK Symbol D 1 D Q Q D EN Q Chapter 3 <21>

Resettable Flip-Flops Inputs: CLK, D, Reset Function: Reset = 1: Q is forced to Reset = : flip-flop behaves as ordinary D flip-flop Symbols D Reset Q r Chapter 3 <22>

Resettable Flip-Flops Two types: Synchronous: resets at the clock edge only Asynchronous: resets immediately when Reset = 1 Asynchronously resettable flip-flop requires changing the internal circuitry of the flip-flop Synchronously resettable flip-flop? Chapter 3 <23>

Settable Flip-Flops Inputs: CLK, D, Set Function: Set = 1: Q is set to 1 Set = : the flip-flop behaves as ordinary D flip-flop Symbols D Set Q s Chapter 3 <25>

Chapter 5 :: Topics Introduction (done) Arithmetic Circuits (done) Number Systems (done) Sequential Building Blocks (done) Memory Arrays (now) Logic Arrays (now) Chapter 3 <26>

Memory Arrays Efficiently store large amounts of data 3 common types: Dynamic random access memory (DRAM) Static random access memory (SRAM) Read only memory (ROM) M-bit data value read/ written at each unique N-bit address Address N Array M Data Chapter 3 <27>

Memory Arrays 2-dimensional array of bit cells Each bit cell stores one bit N address bits and M data bits: 2 N rows and M columns Depth: number of rows (number of words) Width: number of columns (size of word) Array size: depth width = 2 N M Address N Array M Data Address Data Address 2 Array 11 1 1 1 1 1 1 depth 3 1 1 Data width Chapter 3 <28>

Memory Array Example 2 2 3-bit array Number of words: 4 Word size: 3-bits For example, the 3-bit word at address 1 is 1 Address Data Address 2 Array 11 1 1 1 1 1 1 depth 3 1 1 Data width Chapter 3 <29>

Memory Arrays Address 1 124-word x 32-bit Array 32 Data Chapter 3 <3>

Memory Array Bit Cells wordline bit bitline wordline = 1 bit = bitline = wordline = bit = bitline = Z bitline = bitline = wordline = 1 bit = 1 wordline = bit = 1 (a) (b) Chapter 3 <31>

Memory Array Bit Cells wordline bit bitline wordline = 1 bit = bitline = wordline = bit = bitline = Z wordline = 1 bit = 1 bitline = 1 wordline = bit = 1 bitline = Z (a) (b) Chapter 3 <32>

Memory Array Wordline: like an enable single row in memory array read/written corresponds to unique address only one wordline HIGH at once 2:4 Decoder bitline 2 bitline 1 bitline 11 wordline 3 Address 2 1 1 wordline 2 wordline 1 wordline bit = bit = 1 bit = 1 bit = bit = 1 bit = bit = 1 bit = 1 bit = bit = bit = bit = 1 Data 2 Data 1 Data Chapter 3 <33>

Types of Memory Random access memory (RAM): volatile Read only memory (ROM): nonvolatile Chapter 3 <34>

RAM: Random Access Memory Volatile: loses its data when power off Read and written quickly Main memory in your computer is RAM (DRAM) Historically called random access memory because any data word accessed as easily as any other (in contrast to sequential access memories such as a tape recorder) Chapter 3 <35>

ROM: Read Only Memory Nonvolatile: retains data when power off Read quickly, but writing is impossible or slow Flash memory in cameras, thumb drives, and digital cameras are all ROMs Historically called read only memory because ROMs were written at manufacturing time or by burning fuses. Once ROM was configured, it could not be written again. This is no longer the case for Flash memory and other types of ROMs. Chapter 3 <36>

Types of RAM DRAM (Dynamic random access memory) SRAM (Static random access memory) Differ in how they store data: DRAM uses a capacitor SRAM uses cross-coupled inverters Chapter 3 <37>

Robert Dennard, 1932 - Invented DRAM in 1966 at IBM Others were skeptical that the idea would work By the mid-197 s DRAM in virtually all computers Chapter 3 <38>

DRAM Data bits on capacitor Dynamic because the value needs to be refreshed (rewritten) periodically and after read: Charge leakage from the capacitor degrades the value Reading destroys the value wordline bit bitline wordline bit bitline Chapter 3 <39>

DRAM wordline bitline wordline bitline bit = 1 + + bit = Chapter 3 <4>

SRAM wordline bit bitline wordline bitline bitline Chapter 3 <41>

Memory Arrays Review 2:4 Decoder bitline 2 bitline 1 bitline 11 wordline 3 Address 2 1 1 wordline 2 wordline 1 wordline bit = bit = 1 bit = 1 bit = bit = 1 bit = bit = 1 bit = 1 bit = bit = bit = bit = 1 Data 2 Data 1 Data DRAM bit cell: SRAM bit cell: bitline bitline bitline wordline wordline Chapter 3 <42>

ROM: Dot Notation 2:4 Decoder 11 wordline bitline Address 2 1 bit cell containing 1 bitline Data 2 Data 1 Data wordline bit cell containing 1 Chapter 3 <43>

Fujio Masuoka, 1944 - Developed memories and high speed circuits at Toshiba, 1971-1994 Invented Flash memory as an unauthorized project pursued during nights and weekends in the late 197 s The process of erasing the memory reminded him of the flash of a camera Toshiba slow to commercialize the idea; Intel was first to market in 1988 Flash has grown into a $25 billion per year market Chapter 3 <44>

ROM Storage Address 2 2:4 Decoder 11 1 1 Address 11 1 1 Data 1 1 1 1 depth 1 1 Data 2 Data 1 Data width Chapter 3 <45>

ROM Logic Address 2 2:4 Decoder 11 1 1 Data 2 = A 1 A Data 1 = A 1 + A Data = A 1 A Data 2 Data 1 Data Chapter 3 <46>

Example: Logic with ROMs Implement the following logic functions using a 2 2 3-bit ROM: X = AB Y = A + B Z = A B A, B 2 2:4 Decoder 11 1 1 X Y Z Chapter 3 <47>

Example: Logic with ROMs Implement the following logic functions using a 2 2 3-bit ROM: X = AB Y = A + B Z = A B A, B 2 2:4 Decoder 11 1 1 X Y Z Chapter 3 <48>

Logic with Any Memory Array 2:4 Decoder bitline 2 bitline 1 bitline 11 wordline 3 Address 2 1 1 wordline 2 wordline 1 wordline bit = bit = 1 bit = 1 bit = bit = 1 bit = bit = 1 bit = 1 bit = bit = bit = bit = 1 Data 2 = A 1 A Data 1 = A 1 + A Data = A 1 A Data 2 Data 1 Data Chapter 3 <49>

Logic with Memory Arrays Implement the following logic functions using a 2 2 3-bit memory array: X = AB Y = A + B Z = A B Chapter 3 <5>

Logic with Memory Arrays Implement the following logic functions using a 2 2 3-bit memory array: X = AB Y = A + B Z = A B A, B 2 2:4 Decoder 11 1 1 wordline 3 wordline 2 wordline 1 wordline bit = 1 bit = bit = bit = bitline 2 bitline 1 bitline bit = 1 bit = 1 bit = 1 bit = bit = bit = 1 bit = bit = X Y Z Chapter 3 <51>

Logic with Memory Arrays Called lookup tables (LUTs): look up output at each input combination (address) 4-word x 1-bit Array Truth Table A B Y 1 1 1 1 1 A B 2:4 Decoder A 1 1 A 1 11 bit = bit = bit = bit = 1 bitline Y Chapter 3 <52>

Multi-ported Memories Port: address/data pair 3-ported memory 2 read ports (A1/RD1, A2/RD2) 1 write port (A3/WD3, WE3 enables writing) Register file: small multi-ported memory CLK N N A1 A2 WE3 RD1 RD2 M M N M A3 WD3 Array Chapter 3 <53>

Logic Arrays PLAs (Programmable logic arrays) AND array followed by OR array Combinational logic only Fixed internal connections FPGAs (Field programmable gate arrays) Array of Logic Elements (LEs) Combinational and sequential logic Programmable internal connections Chapter 3 <55>

PLAs X = ABC + ABC Y = AB Inputs M AND ARRAY Implicants N OR ARRAY A B C P Outputs OR ARRAY ABC ABC AB AND ARRAY X Y Chapter 3 <56>

PLAs: Dot Notation Inputs M AND ARRAY Implicants N OR ARRAY A B C P Outputs OR ARRAY ABC ABC AB AND ARRAY X Y Chapter 3 <57>

FPGA: Field Programmable Gate Array Composed of: LEs (Logic elements): perform logic IOEs (Input/output elements): interface with outside world Programmable interconnection: connect LEs and IOEs Some FPGAs include other building blocks such as multipliers and RAMs Chapter 3 <58>

General FPGA Layout Chapter 3 <59>

LE: Logic Element Composed of: LUTs (lookup tables): perform combinational logic Flip-flops: perform sequential logic Multiplexers: connect LUTs and flip-flops Chapter 3 <6>

Chapter 8 :: Topics Introduction (now) Memory System Performance Analysis (now) Caches (now) Virtual Memory (now) Memory-Mapped I/O (later) Summary (later) Chapter 3 <66>

Introduction Computer performance depends on: Processor performance Memory system performance Memory Interface CLK CLK MemWrite WE Processor Address WriteData Memory ReadData Chapter 3 <67>

Processor-Memory Gap In prior chapters, assumed access memory in 1 clock cycle but hasn t been true since the 198 s Chapter 3 <68>

Memory System Challenge Make memory system appear as fast as processor Use hierarchy of memories Ideal memory: Fast Cheap (inexpensive) Large (capacity) But can only choose two! Chapter 3 <69>

Memory Hierarchy Technology Price / GB Access Time (ns) Bandwidth (GB/s) Cache SRAM $1, 1 25+ Speed Main Memory DRAM $1 1-5 1 SSD $1 1,.5 Virtual Memory HDD $.1 1,,.1 Capacity Chapter 3 <7>

Locality Exploit locality to make memory accesses fast Temporal Locality: Locality in time If data used recently, likely to use it again soon How to exploit: keep recently accessed data in higher levels of memory hierarchy Spatial Locality: Locality in space If data used recently, likely to use nearby data soon How to exploit: when access data, bring nearby data into higher levels of memory hierarchy too Chapter 3 <71>

Memory Performance Hit: data found in that level of memory hierarchy Miss: data not found (must go to next level) Hit Rate = # hits / # memory accesses = 1 Miss Rate Miss Rate = # misses / # memory accesses = 1 Hit Rate Average memory access time (AMAT): average time for processor to access data AMAT = t cache + MR cache [t MM + MR MM (t VM )] Chapter 3 <72>

Memory Performance Example 1 A program has 2, loads and stores 1,25 of these data values in cache Rest supplied by other levels of memory hierarchy What are the hit and miss rates for the cache? Chapter 3 <73>

Memory Performance Example 2 Suppose processor has 2 levels of hierarchy: cache and main memory t cache = 1 cycle, t MM = 1 cycles What is the AMAT of the program from Example 1? Chapter 3 <75>

Memory Performance Example 2 Suppose processor has 2 levels of hierarchy: cache and main memory t cache = 1 cycle, t MM = 1 cycles What is the AMAT of the program from Example 1? AMAT = t cache + MR cache (t MM ) = [1 +.375(1)] cycles = 38.5 cycles Chapter 3 <76>

Gene Amdahl, 1922- Amdahl s Law: the effort spent increasing the performance of a subsystem is wasted unless the subsystem affects a large percentage of overall performance Co-founded 3 companies, including one called Amdahl Corporation in 197 Chapter 3 <77>

Cache Highest level in memory hierarchy Fast (typically ~ 1 cycle access time) Ideally supplies most data to processor Usually holds most recently accessed data Chapter 3 <78>

Cache Design Questions What data is held in the cache? How is data found? What data is replaced? Focus on data loads, but stores follow same principles Chapter 3 <79>

What data is held in the cache? Ideally, cache anticipates needed data and puts it in cache But impossible to predict future Use past to predict future temporal and spatial locality: Temporal locality: copy newly accessed data into cache Spatial locality: copy neighboring data into cache too Chapter 3 <8>

Cache Terminology Capacity (C): number of data bytes in cache Block size (b): bytes of data brought into cache at once Number of blocks (B = C/b): number of blocks in cache: B = C/b Degree of associativity (N): number of blocks in a set Number of sets (S = B/N): each memory address maps to exactly one cache set Chapter 3 <81>

How is data found? Cache organized into S sets Each memory address maps to exactly one set Caches categorized by # of blocks in a set: Direct mapped: 1 block per set N-way set associative: N blocks per set Fully associative: all cache blocks in 1 set Examine each organization for a cache with: Capacity (C = 8 words) Block size (b = 1 word) So, number of blocks (B = 8) Chapter 3 <82>

Example Cache Parameters C = 8 words (capacity) b = 1 word (block size) So, B = 8 (# of blocks) Ridiculously small, but will illustrate organizations Chapter 3 <83>

Direct Mapped Cache Address 11...111111 11...11111 11...11111 11...1111 11...11111 11...1111 11...1111 11...111 mem[xff...fc] mem[xff...f8] mem[xff...f4] mem[xff...f] mem[xff...ec] mem[xff...e8] mem[xff...e4] mem[xff...e]...11...1...111...11...11...1...11...1...1... mem[x...24] mem[x..2] mem[x..1c] mem[x...18] mem[x...14] mem[x...1] mem[x...c] mem[x...8] mem[x...4] mem[x...] 2 3 Word Main Memory 2 3 Word Cache Chapter 3 <84> Set Number 7 (111) 6 (11) 5 (11) 4 (1) 3 (11) 2 (1) 1 (1) ()

Direct Mapped Cache Hardware Memory Address Tag Set 27 3 Byte Offset V Tag Data 8-entry x (1+27+32)-bit SRAM = 27 32 Hit Data Chapter 3 <85>

Direct Mapped Cache Performance Memory Address # MIPS assembly code addi $t, $, 5 loop: beq $t, $, done lw $t1, x4($) lw $t2, xc($) lw $t3, x8($) addi $t, $t, -1 j loop Tag Data done: Miss Rate =? Tag... Set 1 3 Byte Offset V 1... mem[x...c] 1... mem[x...8] 1... mem[x...4] Set 7 (111) Set 6 (11) Set 5 (11) Set 4 (1) Set 3 (11) Set 2 (1) Set 1 (1) Set () Chapter 3 <86>

Direct Mapped Cache Performance Memory Address # MIPS assembly code Tag... addi $t, $, 5 loop: beq $t, $, done lw $t1, x4($) lw $t2, xc($) lw $t3, x8($) addi $t, $t, -1 j loop Tag Data done: Miss Rate = 3/15 = 2% Set 1 3 Byte Offset V 1... mem[x...c] 1... mem[x...8] 1... mem[x...4] Temporal Locality Compulsory Misses Set 7 (111) Set 6 (11) Set 5 (11) Set 4 (1) Set 3 (11) Set 2 (1) Set 1 (1) Set () Chapter 3 <87>

Direct Mapped Cache: Conflict Memory Address # MIPS assembly code Tag...1 addi $t, $, 5 loop: beq $t, $, done lw $t1, x4($) lw $t2, x24($) addi $t, $t, -1 j loop done: Set 1 3 Byte Offset V 1 Tag... Data mem[x...4] mem[x...24] Set 7 (111) Set 6 (11) Set 5 (11) Set 4 (1) Set 3 (11) Set 2 (1) Set 1 (1) Set () Miss Rate =? Chapter 3 <88>

1 N-Way Set Associative Cache Memory Address Tag Set 28 2 Byte Offset V Tag Way 1 Way Data V Tag Data 28 32 28 32 = = Hit 1 Hit Hit 1 32 Hit Data Chapter 3 <9>

N-Way Set Associative Performance # MIPS assembly code addi $t, $, 5 loop: beq $t, $, done lw $t1, x4($) lw $t2, x24($) addi $t, $t, -1 j loop done: Miss Rate =? Way 1 Way V Tag Data V Tag Data Set 3 Set 2 Set 1 Set Chapter 3 <91>

N-Way Set Associative Performance # MIPS assembly code addi $t, $, 5 loop: beq $t, $, done lw $t1, x4($) lw $t2, x24($) addi $t, $t, -1 j loop done: Miss Rate = 2/1 = 2% Associativity reduces conflict misses Way 1 Way V Tag Data V Tag Data 1...1 mem[x...24] 1... mem[x...4] Set 3 Set 2 Set 1 Set Chapter 3 <92>

Fully Associative Cache V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data V Tag Data Reduces conflict misses Expensive to build Chapter 3 <93>

1 1 11 Spatial Locality? Increase block size: Block size, b = 4 words C = 8 words Direct mapped (1 block per set) Number of blocks, B = 2 (C/b = 8/4 = 2) Memory Address Block Byte Tag Set Offset Offset 27 2 V Tag 27 Data 32 32 32 32 Set 1 Set = 32 Hit Data Chapter 3 <94>

1 1 11 Cache with Larger Block Size Memory Address Block Byte Tag Set Offset Offset 27 2 V Tag 27 Data 32 32 32 32 Set 1 Set = 32 Hit Data Chapter 3 <95>

Direct Mapped Cache Performance loop: done: addi $t, $, 5 beq $t, $, done lw $t1, x4($) lw $t2, xc($) lw $t3, x8($) addi $t, $t, -1 j loop Miss Rate =? Chapter 3 <96>

1 1 11 Direct Mapped Cache Performance loop: done: Memory Address addi $t, $, 5 beq $t, $, done lw $t1, x4($) lw $t2, xc($) lw $t3, x8($) addi $t, $t, -1 j loop Tag Block Byte Set Offset Offset... 11 27 2 V Tag 1... mem[x...c] 27 Miss Rate = 1/15 = 6.67% Larger blocks reduce compulsory misses through spatial locality Data mem[x...8] mem[x...4] mem[x...] 32 32 32 32 Set 1 Set = 32 Hit Data Chapter 3 <97>

Cache Organization Recap Capacity: C Block size: b Number of blocks in cache: B = C/b Number of blocks in a set: N Number of sets: S = B/N Organization Number of Ways (N) Direct Mapped 1 B N-Way Set Associative 1 < N < B B / N Fully Associative B 1 Number of Sets (S = B/N) Chapter 3 <98>

Capacity Misses Cache is too small to hold all data of interest at once If cache full: program accesses data X & evicts data Y Capacity miss when access Y again How to choose Y to minimize chance of needing it again? Least recently used (LRU) replacement: the least recently used block in a set evicted Chapter 3 <99>

Types of Misses Compulsory: first time data accessed Capacity: cache too small to hold all data of interest Conflict: data of interest maps to same location in cache Miss penalty: time it takes to retrieve a block from lower level of hierarchy Chapter 3 <1>

LRU Replacement # MIPS assembly lw $t, x4($) lw $t1, x24($) lw $t2, x54($) Way 1 Way V U Tag Data V Tag Data Set 3 (11) Set 2 (1) Set 1 (1) Set () Chapter 3 <11>

LRU Replacement # MIPS assembly lw $t, x4($) lw $t1, x24($) lw $t2, x54($) Way 1 Way (a) V U Tag Data V Tag Data 1...1 mem[x...24] 1... mem[x...4] Way 1 Way Set 3 (11) Set 2 (1) Set 1 (1) Set () (b) V U Tag Data V Tag 1 1...1 mem[x...24] 1...11 Data mem[x...54] Chapter 3 <12> Set 3 (11) Set 2 (1) Set 1 (1) Set ()

Cache Summary What data is held in the cache? Recently used data (temporal locality) Nearby data (spatial locality) How is data found? Set is determined by address of data Word within block also determined by address In associative caches, data could be in one of several ways What data is replaced? Least-recently used way in the set Chapter 3 <13>

Miss Rate Trends Bigger caches reduce capacity misses Greater associativity reduces conflict misses Adapted from Patterson & Hennessy, Computer Architecture: A Quantitative Approach, 211 Chapter 3 <14>

Miss Rate Trends Bigger blocks reduce compulsory misses Bigger blocks increase conflict misses Chapter 3 <15>

Multilevel Caches Larger caches have lower miss rates, longer access times Expand memory hierarchy to multiple levels of caches Level 1: small and fast (e.g. 16 KB, 1 cycle) Level 2: larger and slower (e.g. 256 KB, 2-6 cycles) Most modern PCs have L1, L2, and L3 cache Chapter 3 <16>

Intel Pentium III Die Chapter 3 <17>

Virtual Memory Gives the illusion of bigger memory Main memory (DRAM) acts as cache for hard disk Chapter 3 <18>

Memory Hierarchy Technology Price / GB Access Time (ns) Bandwidth (GB/s) Cache SRAM $1, 1 25+ Speed Main Memory DRAM $1 1-5 1 SSD $1 1,.5 Virtual Memory HDD $.1 1,,.1 Capacity Physical Memory: DRAM (Main Memory) Virtual Memory: Hard drive Slow, Large, Cheap Chapter 3 <19>

Hard Disk Magnetic Disks Read/Write Head Takes milliseconds to seek correct location on disk Chapter 3 <11>

Virtual Memory Virtual addresses Programs use virtual addresses Entire virtual address space on a hard drive Subset of virtual address data in DRAM CPU translates virtual addresses into physical addresses (DRAM addresses) Data not in DRAM fetched from hard drive Memory Protection Each program has own virtual to physical mapping Two programs can use same virtual address for different data Programs don t need to be aware others are running One program (or virus) can t corrupt memory used by another Chapter 3 <111>

Cache/Virtual Memory Analogues Cache Block Block Size Block Offset Miss Tag Virtual Memory Page Page Size Page Offset Page Fault Virtual Page Number Physical memory acts as cache for virtual memory Chapter 3 <112>

Virtual Memory Definitions Page size: amount of memory transferred from hard disk to DRAM at once Address translation: determining physical address from virtual address Page table: lookup table used to translate virtual addresses to physical addresses Chapter 3 <113>

Virtual & Physical Addresses Most accesses hit in physical memory But programs have the large capacity of virtual memory Chapter 3 <114>

Address Translation Chapter 3 <115>

Virtual Memory Example System: Virtual memory size: 2 GB = 2 31 bytes Physical memory size: 128 MB = 2 27 bytes Page size: 4 KB = 2 12 bytes Chapter 3 <116>

Virtual Memory Example System: Virtual memory size: 2 GB = 2 31 bytes Physical memory size: 128 MB = 2 27 bytes Page size: 4 KB = 2 12 bytes Organization: Virtual address: 31 bits Physical address: 27 bits Page offset: 12 bits # Virtual pages = 2 31 /2 12 = 2 19 (VPN = 19 bits) # Physical pages = 2 27 /2 12 = 2 15 (PPN = 15 bits) Chapter 3 <117>

Virtual Memory Example 19-bit virtual page numbers 15-bit physical page numbers Chapter 3 <118>

Virtual Memory Example What is the physical address of virtual address x247c? Chapter 3 <119>

Virtual Memory Example What is the physical address of virtual address x247c? VPN = x2 VPN x2 maps to PPN x7fff 12-bit page offset: x47c Physical address = x7fff47c Chapter 3 <12>

How to perform translation? Page table Entry for each virtual page Entry fields: Valid bit: 1 if page in physical memory Physical page number: where the page is located Chapter 3 <121>

Page Table Example Virtual Address Virtual Page Number x2 19 Page Offset 47C 12 VPN is index into page table V 1 x 1 x7ffe 1 x1 1 x7fff Hit Physical Address Physical Page Number x7fff Page Table 15 12 47C Chapter 3 <122>

Page Table Example 1 What is the physical address of virtual address x5f2? V Physical Page Number 1 x 1 x7ffe 1 x1 1 x7fff Page Table Chapter 3 <123>

Page Table Example 1 Virtual Address What is the physical address of virtual address x5f2? VPN = 5 Entry 5 in page table VPN 5 => physical page 1 Physical address: x1f2 Virtual Page Number x5 19 Page Offset F2 12 V 1 x 1 x7ffe 1 x1 1 x7fff Hit Physical Address Physical Page Number Chapter 3 <124> Page Table 15 12 x1 F2

Page Table Example 2 Virtual Address What is the physical address of virtual address x73e? Virtual Page Number x7 19 Page Offset 3E V Physical Page Number 1 x 1 x7ffe 1 x1 1 x7fff Page Table Hit 15 Chapter 3 <125>

Page Table Example 2 Virtual Address What is the physical address of virtual address x73e? VPN = 7 Entry 7 is invalid Virtual page must be paged into physical memory from disk Virtual Page Number x7 19 Page Offset 3E V Physical Page Number 1 x 1 x7ffe 1 x1 1 x7fff Page Table Hit 15 Chapter 3 <126>

Page Table Challenges Page table is large usually located in physical memory Load/store requires 2 main memory accesses: one for translation (page table read) one to access data (after translation) Cuts memory performance in half Unless we get clever Chapter 3 <127>

Translation Lookaside Buffer (TLB) Small cache of most recent translations Reduces # of memory accesses for most loads/stores from 2 to 1 Chapter 3 <128>

TLB Page table accesses: high temporal locality Large page size, so consecutive loads/stores likely to access same page TLB Small: accessed in < 1 cycle Typically 16-512 entries Fully associative > 99 % hit rates typical Reduces # of memory accesses for most loads/stores from 2 to 1 Chapter 3 <129>

1 Example 2-Entry TLB Virtual Address Virtual Page Number x2 19 Page Offset 47C 12 Entry 1 Entry V Virtual Page Number Physical Page Number 1 x7fffd x 1 x2 x7fff V Virtual Page Number Physical Page Number 19 15 19 15 TLB = = Hit 1 Hit Hit 1 Hit Physical Address 15 x7fff 47C 12 Chapter 3 <13>

Memory Protection Multiple processes (programs) run at once Each process has its own page table Each process can use entire virtual address space A process can only access physical pages mapped in its own page table Chapter 3 <131>

Virtual Memory Summary Virtual memory increases capacity A subset of virtual pages in physical memory Page table maps virtual pages to physical pages address translation A TLB speeds up address translation Different page tables for different programs provides memory protection Chapter 3 <132>