Lecture 19: Survey of Modern VMs + a Decomposition of Meltdown. James C. Hoe Department of ECE Carnegie Mellon University

Transcription

1 Lecture 19: Survey of Modern VMs + a Decomposition of Meltdown James C. Hoe Department of ECE Carnegie Mellon University S18 L19 S1, James C. Hoe, CMU/ECE/CALCM, 2018

2 Housekeeping Your goal today see the many realizations of VM, focusing on deviation from textbook conceptual norms put everything you learned together in Meltdown Notices Midterm 4/9 in class; covers Lectures 10~19 HW 4, due on Friday Lab 4, status check next week Prof. Ghose will give ILP lecture on Thursday Readings start on P&H Ch S18 L19 S2, James C. Hoe, CMU/ECE/CALCM, 2018

3 EA, VA and PA (IBM Power view) 64 bit EA 0 divided into X fixed size segments 40~50 bit PA divided into W pages (Z>>W) 64 bit EA 1 divided into X fixed size segments segmented EA: private, contiguous + sharing S18 L19 S3, James C. Hoe, CMU/ECE/CALCM, ~90 bit VA divided into Y segments (Y>>X); also divided as Z pages (Z>Y) swap disk divided into V pages (Z>>V, V>W) demand paged VA: size of swap, speed of DRAM

4 EA, VA and PA (almost everyone else) EA 0 with unique ASID=0 PA divided into W pages (Z>>W) EA i with unique ASID=i swap disk divided into V pages (Z>>V, V>>W) S18 L19 S4, James C. Hoe, CMU/ECE/CALCM, 2018 VA divided into N address spaces indexed by ASID; also divided as Z pages (Z>>N) how do processes share pages?

5 B. Jacob and T. Mudge, Virtual Memory in Contemporary Processors, IEEE Micro, S18 L19 S5, James C. Hoe, CMU/ECE/CALCM, 2018

6 SPARC V9 PTE/TLB Entry 64 bit VA + context ID implementation can choose not to map high order bits (require sign extension in unmapped bits) e.g., UltraSPARC 1 maps only lower 44 bits PA space size set by implementation 64 entry fully associative I TLB and D TLB g context 13 VA<63:13> 51 Tag 64 v size nfo IE Soft diag PA<40:13> Soft L CP CV e p w g PTE 64 Details fixed and exposed by privileged ISA S18 L19 S6, James C. Hoe, CMU/ECE/CALCM, 2018

7 context S18 L19 S7, James C. Hoe, CMU/ECE/CALCM, 2018 SPARC TLB Miss Handling 32 bit V8 used a 3 level hierarchical page table for HW MMU page table walk context table descriptors +VA [31:24] L1 Table: L2 Table: 256 +VA [23:18] 64 +VA [17:12] descriptors (1024 byte) descriptors (256 byte) 64 bit V9 switched to Translation Storage Buffer a software managed, direct mapped cache of PTEs (think hashed page table) HW assisted address generation on a TLB miss TLB miss handler (SW) searches TSB. If TSB misses, a slower TSB miss handler takes over OS can use any page table structure after TSB L3 Table: 64 PTEs (256 byte)

8 IBM PowerPC (32 bit) segments 256MB regions seg# 4 seg offset 16 page offset 12 EA entry segment table Protection seg ID 24 seg offset 16 page offset 12 VA way ITLB and DTLB VM+paging S18 L19 S8, James C. Hoe, CMU/ECE/CALCM, 2018 PPN 20 page offset 12 PA bit PowerPC = 64 bit EA > 80 bit VA 64 bit PA How many segments in 64 bit EA?

9 IBM PowerPC Hashed Page Table VPN 40 hash function Hashed Page Table table base + HW table walk VPN hashes into a PTE group (PTEG) of 8 8 PTEs searched sequentially for tag match if not found in first PTEG search a second PTEG if not found in 2 nd PTEG, trap to software handler Hashed table structure also used for 64 bit EA VA S18 L19 S9, James C. Hoe, CMU/ECE/CALCM, PTE s per group (recommend at least N PTEG s for a system with 2N physical pages)

10 64 bit VA MIPS R10K top 2 bits set kernel/supervisor/user mode additional bits set cache and translation behavior bit not translate at all (holes and repeats in the VA??) 8 bit ASID (address space ID) distinguishes between processes 40 bit PA Translation 64 bit VA and 8 bit ASID 40 bit PA 1 GB mapped (kseg) 0.5 GB unmapped uncached 0.5 GB unmapped cached bottom 2 GB mapped (normal) simplified example from 32 bit VA in R2000/ S18 L19 S10, James C. Hoe, CMU/ECE/CALCM, 2018

11 MIPS TLB 64 entry fully associative unified TLB Each entry maps 2 consecutive VPNs to independent respective PPNs Software TLB miss handling (exotic at the time) 7 instruction page table walk in the best case TLB Write Random: chooses a random entry for TLB replacement OS can exclude low TLB entries from replacement (some translations must not miss) TLB entry N: noncacheable D: dirty (write enable!!) V: valid G: ignore ASID S18 L19 S11, James C. Hoe, CMU/ECE/CALCM, 2018 VPN 20 ASID PPN 20 ndvg 0 8

12 MIPS Bottom Up Hierarchical Table TLB miss vectors to a SW handler page table organization is not hardcoded in ISA ISA favors a chosen reference page table scheme by providing optional hardware assistance Bottom Up Table start with 2 level hierarchical table (32 bit case) allocate all L2 tables for all VA pages (empty or not) linearly in the mapped kseg space VPN is index into this linear table in VA This table scales with VA size!! Is this okay? S18 L19 S12, James C. Hoe, CMU/ECE/CALCM, 2018

13 Bottom Up Table Walk VPN PO VA on TLB Miss, trap PTEBase mem load VPN 0s whose address VA of PTE space? (generated automatically by HW after TLB miss) PPN status PTE loaded from mem Can this load miss in the TLB? What happens if it misses? S18 L19 S13, James C. Hoe, CMU/ECE/CALCM, 2018 notice translation also eats up TLB entries!

14 User TLB Miss Handling mfc0 k0,tlbcxt mfc0 k1,epc lw k0,0(k0) mtc0 k0,entry_lo tlbwr j k1 rfe # move the contents of TLB # context register into k0 # move PC of faulting memory # instruction into k1 # load thru address that was # in TLB context register # move the loaded value (a PTE) # into the EntryLo register # write PTE into the TLB # at a random slot number # jump to PC of faulting # load instruction to retry # restore priviledge (in delay slot) S18 L19 S14, James C. Hoe, CMU/ECE/CALCM, 2018

15 S18 L19 S15, James C. Hoe, CMU/ECE/CALCM, 2018 HP PA RISC: PID and AID 2 level: 64b EA 96b VA (global) 64b PA Variable sized segmented EA VA translation Rights based access control user controls segment registers (user can generate any VA it wants!!) in contrast, everyone else controls translation to control what VA can be reached from a process each virtual page has an access ID (AID) assigned by OS each process has 8 active protection IDs (PIDs) in privileged HW registers controlled by OS a process can access a page only if one of the 8 PIDs matches the page s AID

16 Intel Two level address translation: segmented EA global VA PA User private 48 bit EA 16 bit SN (implicit) + 32 bit SO 6 user controlled registers hold active SNs; selected according to usage: code, data, stack, etc Global 32 bit VA 20 bit VPN + 12 bit PO An implementation defined paged PA space What is very odd about this? S18 L19 S16, James C. Hoe, CMU/ECE/CALCM, 2018

17 Living with the mistake 32 bit global VA too small to share by processes per process EA space oddly bigger than VA space until 1990, no one cared DOS and Windows Later multitasking OS ignore segment protection time multiplex **global** VA space for use by 1 process at a time code, data, stack segments always map to entire VA space, 0~(2 32 1) set MMU to use a different table on context switch BUT! TLB for VA translation doesn t have ASID; must flush TLB on context switch Later IA32e added PCID to TLB as fix S18 L19 S17, James C. Hoe, CMU/ECE/CALCM, 2018

18 Meltdown in Terms How to know the value at a memory location without permission? S18 L19 S18, James C. Hoe, CMU/ECE/CALCM, 2018

19 VA to PA Translation Flow Chart EA TLB lookup ISA says can't read without permission no hit yes 10~100 pclk PT walk page in DRAM found page on disk page fault update TLB demand paging 10 msec don t exist seg fault now what? no protection violation protection check okay yes PA to cache S18 L19 S19, James C. Hoe, CMU/ECE/CALCM, 2018

20 How should VM and Cache Interact? CPU CPU CPU physical TLB cache lower hier. VA PA S18 L19 S20, James C. Hoe, CMU/ECE/CALCM, 2018 virtual cache VA hybrid?? cache TLB VA PA TLB Actually you can read PA without permission; ISA (as an lower abstraction) lower only care you hier. can t hier. see the read value Only a question for L1 caches

21 Flushing a Pipeline privileged mode t 0 t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 t 10 IF I 0 I 1 I 2 I 3 I 4 bub bub bub I h I h+1 I h+2 ID I 0 I 1 I 2 I 3 bub bub bub bub I h I h+1 EX S18 L19 S21, James C. Hoe, CMU/ECE/CALCM, 2018 I 0 I 1 I 2 I 3 bub bub bub bub bub bub I h can MEM read without permission I 0 I bub bub bub 1 I 2 bub can even use read value in WB I 0 I 1 I bub bub bub dependent instructions 2 as Kill long faulting as at the and end younger can t inst; drain older inst see Don t any start of it handler until faulting inst. is oldest 100s of speculative instructions Better yet, don t start handler until pipeline is empty in flight in modern OOO CPUs Better to be safe than to be fast

22 Key Idea 3: Inter Model Compatibility a valid program whose logic will not depend implicitly upon time of execution and which runs upon configuration A, will also run on configuration B if the a fundamental tenet that ISA latter includes at least the required storage, at least the does not care about time required I/O devices. Invalid programs not constrained to yield same result invalid ==violating architecture manual exceptions are architecturally defined The King of Binary Compatibility: Intel x86, IBM 360 stable software base and ecosystem performance scalability [Amdahl, Blaauw and Brooks, 1964] S18 L19 S22, James C. Hoe, CMU/ECE/CALCM, 2018

23 What cache is in your computer? How to figure out what cache configuration is in your computer Cache invisible architecturally, but performance capacity side effect (C), associativity easily (a), detectable and block size using timer (B) number of levels The presence or lack of a cache should not be detectable by functional behavior of software But you could tell if you measured execution time to infer the number of cache misses Infer read value without seeing by running code to cause hit/miss based on unseen value S18 L19 S23, James C. Hoe, CMU/ECE/CALCM, 2018

24 64 bit virtual address MIPS R10K Read top addr 2 bits Y+C, set kernel/supervisor/user Y+2C, Y+3C... so addr mode Y is additional not in cache; bits then set cache attempt and to translation execute: behavior I 1 : lw t0, 0(r X ) bit I 2 : andinot t0, translate t0, at 0x1 all I (holes 3 : sli t0, t0, log and repeats in the 2 (blocksize) VA??) I 4 : add t0, t0, r Y I 5 : lw x0, 0(t0) 8 bit ASID (address space ID) distinguishes between processes I 1 is an exception so I 1 ~I 5 not observed architecturally; nevertheless addr Y is cached if LSB of mem[x] is 0 40 bit physical address Translation 64 bit VA and 8 bit ASID 40 bit PA 1 GB mapped (kseg) X: 0.5 GB unmapped uncached 0.5 GB unmapped cached Y: bottom 2 GB mapped (normal) simplified example from 32 bit VA in R2000/ S18 L19 S24, James C. Hoe, CMU/ECE/CALCM, 2018

25 Control Speculation: PC+4 t 0 t 1 t 2 t 3 t 4 t 5 Inst h IF PC ID ALU MEM Inst i Inst j IF PC+4 ID IF PC+8 ALU ID Inst k IF target Train BTB so the previous executes as wrong path first opportunity to decode Inst h should we architecturally correct now? nothing Inst h branch condition and target illegal happened!! evaluated in ALU Inst h is a taken branch S18 L19 S25, James C. Hoe, CMU/ECE/CALCM, 2018 When inst h branch resolves branch target (Inst k ) is fetched flush instructions fetched since inst h ( wrong path )

26 Idempotency and Side effects Meltdown vulnerability not a bug but a purposeful Loading from real memory location M[A] should performance return most optimization recent value permitted stored to by M[A] ISA Same writing issue doesn t M[A] once arise is with the same MMIO because as writing M[A] ISA with disallow same spurious value multiple read if times TLB says in a uncacheable row Architects and architects need to be more paranoid reading M[A] multiple times returns same value This is why memory caching works!! LW/SW to mmap locations can have side effects reading/writing mmap location can imply commands and other state changes consider a FIFO example SW to 0xffff0000 pushes value LW from 0xffff0000 returns popped value S18 L19 S26, James C. Hoe, CMU/ECE/CALCM, xffff0000 FIFO What happens if 0xffff0000 is cached?

27 S18 L19 S27, James C. Hoe, CMU/ECE/CALCM, 2018 Midterm2 Covers lectures (L10~L19), HW, projects, assigned readings (from textbooks and papers) Types of questions freebies: remember the materials >> probing: understand the materials << applied: apply the materials in original interpretation **55 minutes, 55 points** point values calibrated to time needed closed book, one 8½x11 in 2 hand written cribsheet no electronics use pencil or black/blue ink only