COSC3330 Computer Architecture Lecture 20. Virtual Memory

Transcription

1 COSC3330 Computer Architecture Lecture 20. Virtual Memory Instructor: Weidong Shi (Larry), PhD Computer Science Department University of Houston

2 Virtual Memory Topics

3 Reducing Cache Miss Penalty (#2) Use multiple levels of caches Primary cache (Level-1 or L1) attached to CPU Small, but fast Level-2 (L2) cache services misses from primary cache Larger, slower, but still faster than main memory Level-3 (L3) cache services misses from L2 cache Largest, slowest, but still faster than main memory Main memory services L3 cache misses Advancement in semiconductor technology allows enough room on the die for L1, L2, and, L3 caches L2 and L3 caches are typically unified, meaning that it holds both instructions and data 3

4 Core i7 Example CPU Core Reg File Core i7.. CPU Core Reg File L1 I$ (32KB) L1 D$ (32KB) L1 I$ (32KB) L1 D$ (32KB) L2 Cache (256KB) L2 Cache (256KB) L3 Cache (8MB) - Shared 4

5 Given CPU base CPI = 1 Clock frequency = 4GHz Multilevel Cache Example Miss rate/instruction = 2% Main memory access time = 100ns With just L1 cache L1 $ access time = 0.25 ns (1 cycle) L1 miss penalty CPI 100ns/0.25ns = 400 cycles = 9 CPI (= 98% x 1 CPI + 2% x ( )) Now add L2 cache L2 $ access time = 5ns Global miss rate to main memory = 0.5% L1 miss with L2 hit L1 miss penalty = 5ns/0.25ns = 20 cycles L1 miss with L2 miss Extra penalty = 400 cycles CPI = 3.4 CPI 5 (= x (75% x % x (20+400))) Performance ratio = 9/3.4 = 2.6 Addition of L2 cache in this example achieves 2.6x speedup

6 Multilevel Cache Design Considerations Design considerations for L1 and L2 caches are very different L1 should focus on minimizing hit time in support of a shorter clock cycle Smaller in size and may use smaller block sizes L2 (L3) should focus on reducing miss rate to reduce the penalty of long main memory access times Larger in size and may use larger block sizes Higher levels of associativity 6

7 Multilevel Cache Design Considerations The miss penalty of the L1 cache is significantly reduced by the presence of L2 and L3 caches So, L1 can be smaller (i.e., faster) but have a higher miss rate For the L2 (L3) cache, the hit time is less important than the miss rate The L2 (L3) hit time determines L1 s miss penalty

8 Views of Memory Real machines have limited amounts of memory 640KB? A few GB? (This laptop = 4GB) Programmer doesn t want to be bothered Do you think, oh, this computer only has 128MB so I ll write my code this way What happens if you run on a different machine?

9 Programmer s View Example 32-bit memory When programming, you don t care about how much real memory there is Even if you use a lot, memory can always be paged to disk Kernel Text Data 0-2GB Heap AKA Virtual Addresses Stack 4GB

10 Programmer s View Really Program s View Each program/process gets its own 4GB space Kernel Kernel Text Data Heap Stack Kernel Text Data Heap Text Data Heap Stack Stack

11 CPU s View At some point, the CPU is going to have to loadfrom/store-to memory all it knows is the real, A.K.A. physical memory which unfortunately is often < 4GB and is pretty much never 4GB per process

12 Absolute Addresses EDSAC, early 50 s Only one program ran at a time, with unrestricted access to entire machine (RAM + I/O devices) Addresses in a program depended upon where the program was to be loaded in memory But it was more convenient for programmers to write location-independent subroutines How could location independence be achieved? Linker and/or loader modify addresses of subroutines and callers when building a program memory image

13 Physical Memory Dynamic Address Translation Motivation In the early machines, I/O operations were slow and each word transferred involved the CPU Higher throughput if CPU and I/O of 2 or more programs were overlapped. How? multiprogramming Location-independent programs Programming and storage management ease => need for a base register Protection Independent programs should not affect each other inadvertently => need for a bound register prog1 prog2

14 Main Memory Simple Base and Bound Translation Bound Register Segment Length Bounds Violation? Load X Effective Address + Physical Address current segment Program Address Space Base Register Base Physical Address Base and bounds registers are visible/accessible only when processor is running in the supervisor mode

15 Memory Fragmentation user 1 user 2 user 3 OS Space 16K 24K 24K 32K Users 4 & 5 arrive user 1 user 2 user 4 user 3 OS Space 16K 24K 16K 8K 32K Users 2 & 5 leave user 1 user 4 user 3 OS Space 16K 24K 16K 8K 32K free 24K user 5 24K 24K As users come and go, the storage is fragmented. Therefore, at some stage programs have to be moved around to compact the storage.

16 Pages Memory is divided into pages, which are nothing more than fixed sized and aligned regions of memory Typical size: 4KB/page (but not always) Page Page Page Page 3

17 Page Table Map from virtual addresses to physical locations 0K 4K 8K 12K Virtual Addresses Page Table implements this V P mapping 0K 4K 8K 12K 16K 20K 24K 28K Physical Addresses Physical Location may include hard-disk

18 0K 4K 8K 12K 0K 4K 8K 12K Page Tables Physical Memory 0K 4K 8K 12K 16K 20K 24K 28K

19 Page Fault Handler When the referenced page is not in DRAM: The missing page is located (or created) It is brought in from disk, and page table is updated Another job may be run on the CPU while the first job waits for the requested page to be read from disk If no free pages are left, a page is swapped out Pseudo-LRU replacement policy Since it takes a long time to transfer a page (msecs), page faults are handled completely in software by the OS

20 Where Should Page Tables Reside? Space required by the page tables (PT) is proportional to the address space, number of users,... Space requirement is large Idea: Keep PTs in the main memory needs one reference to retrieve the page base address and another to access the data word doubles the number of memory references!

21 Need for Translation Virtual Address Virtual Page Number 0xFC51908B Page Offset Physical Address 0xFC519 Page Table 0x00152 Main Memory 0x B

22 Simple Page Table Flat organization One entry per page Entry contains physical page number (PPN) or indicates page is on disk or invalid Also meta-data (e.g., permissions, dirtiness, etc.) Total size? One entry per page

23 Multi-Level Page Tables Virtual Page Number Level 1 Level 2 Page Offset Physical Page Number

24 Virtual Address 31 Multi-Level Page Tables p1 p2 offset 0 10-bit L1 index Root of the Current Page Table 10-bit L2 index p1 p2 offset (Processor Register) Level 1 Page Table page in primary memory page in secondary memory PTE of a nonexistent page Level 2 Page Tables Data Pages

25 Choosing a Page Size Page size inversely proportional to page table overhead Large page size permits more efficient transfer to/from disk vs. many small transfers Think NFS Small page leads to less fragmentation Big page likely to have more bytes unused

26 CPU Memory Access Program deals with virtual addresses Load R1 = 0[R2] On memory instruction 1. Compute virtual address (0[R2]) 2. Compute virtual page number 3. Compute physical address of VPN s page table entry 4. Load* mapping 5. Compute physical address 6. Do the actual Load* from memory Could be more depending On page table organization

27 Impact on Performance? Every time you load/store, the CPU must perform two (or more) accesses! Even worse, every fetch requires translation of the PC! Observation: Once a virtual page is mapped into a physical page, it ll likely stay put for quite some time

28 Idea: Caching! Not caching of data, but caching of translations 0K 4K 8K 12K Virtual Addresses 0K 4K 8K 12K 16K 20K 24K 28K Physical Addresses VPN X 2 4 PPN 4

29 Translation Cache: TLB TLB = Translation Look-aside Buffer Virtual Address TLB Physical Address Cache Data Cache Tags Hit? If TLB hit, no need to do page table lookup from memory Note: data cache accessed by physical addresses now

30 PAPT Cache Previous slide showed Physically-Addressed Physically-Tagged cache Sometimes called PIPT (I=Indexed) Con: TLB lookup and cache access serialized Caches already take > 1 cycle Pro: cache contents valid so long as page table not modified

31 Virtually Addressed Cache Virtual Address Cache Data Cache Tags (VIVT: vitually indexed, virtually tagged) Hit? On Cache Miss TLB Physical Address To L2 Pro: latency no need to check TLB Con: Cache must be flushed on process change

32 Virtually Indexed Physically Tagged Virtual Address Cache Data Cache Tags Physical Tag = Hit? TLB Physical Address Pro: latency TLB parallelized Pro: don t need to flush $ on process swap Big page size can help here Con: Limit on cache indexing (can only use bits not from the VPN/PPN)

33 Often fully-associative TLB Design For latency, this means few entries However, each entry is for a whole page Ex. 32-entry TLB, 4KB page how big of working set while avoiding TLB misses? If many misses: Increase TLB size (latency problems) Increase page size (fragmenation problems)

34 Process Changes With physically-tagged caches, don t need to flush cache on context switch But TLB is no longer valid! Add process ID to translation Only flush TLB when Recycling PIDs PID:0 VPN:8 PID:1 VPN: PPN: 28 PPN: 44

35 TLB Valid VPN PPN Example Page Table Valid PPN or in Disk 5 disk 12 disk disk 4 VPN Hit or Miss or Page Fault Miss Hit Hit Page Fault Hit Page Fault