5-level paging in Xen

Size: px

Start display at page:

Download "5-level paging in Xen"

Patience Thornton
6 years ago
Views:

1 5-level paging in Xen Yu Zhang 1

2 Agenda Summary of current architecture and motivation 5 level paging introduction Xen design consideration overview 2

3 Current architecture and motivation 64-bit mode in IA32-e Linear address(la) width: 48-bit LA space: 256TB Physical address(pa) width: 46-bit at most PA space: 64TB 64TB is big... enough? 3

4 Current architecture and motivation Industry trend In-Memory Databases (IMDB). Large-scale systems reaching the limit of 64TB. CPU DRAM New memory technology 2 level system memory based on Intel 3D XPoint: - considerably faster than NAND flash; - more endurable than NAND flash; - denser than DRAM with lower cost. Intel DIMMs (3D Xpoint based) Intel Optane SSDs (3D Xpoint based) NAND SSD HDD 4

5 Current architecture and motivation OS requirements 2 more LA bits than PA bits one extra bit to divide the linear address space in half : user/kernel spaces; one more extra bit to provide a direct mapping in kernel linear space for whole physical memory to avoid the highmem/lowmem problem. Conclusion: With PA width greater than 46 bit foreseeable, LA width greater than 48 bit is required, hence 5 level paging. 5

6 5 level paging overview New paging mode in IA32-e: 5 level paging (AKA LA57) Paging Mode CR0.PG CR4.PAE IA32_EFER.LME CR4.LA57 PA Width(note) LA Width IA32-e 4 level paging Up to IA32-e 5 level paging Up to Note: PA width is always bounded by CPUID.MAXPHYADDR which is no greater than 46 on existing processors; can be extended to up to 52 on processors which have 5 level paging. 6

Current architecture and motivation Canonical standard 48-bit canonical 57-bit canonical 48-bit canonical (upper half) FFFFFFFFFFFFFFFF FFFF800000000000 FFFF7FFFFFFFFFFF FFFF7FFFFFFFFFFF 57-bit

7 Current architecture and motivation Canonical standard 48-bit canonical 57-bit canonical 48-bit canonical (upper half) FFFFFFFFFFFFFFFF FFFF FFFF7FFFFFFFFFFF FFFF7FFFFFFFFFFF 57-bit canonical (upper half) FF FEFFFFFFFFFFFFFF Canonical check rule paging-mode canonicality CPU canonicality (for special register loads due to virtualization concerns) 48-bit canonical hole FFFFFFFFFFFFFF 57-bit canonical hole bit canonical (lower half) 48-bit canonical (lower half) 00007FFFFFFFFFFF

8 5 level paging overview Together with new PA width comes: new reserved bit mask in paging structure entries; an new EPT mode: 5 level EPT; impacts on other features, e.g. IOMMU etc. 8

9 5 level paging overview PML 5 PML 4 Directory Ptr Directory Table Offset 0 4-Kbyte Page Physical Addr 9 9 PDPTE 40 PDE with PS =0 Page -Directory 40 9 PTE 40 Page Table PML 4E 40 Linear-Address Translation Using 5-Level Paging CR3[51:12] contains physical address of PML5 table; PML5E linear adress bits 56:48 select an PML5E; PML5E contains physical address of PML4 table; 40 page walk follows using linear address bits 47:0. CR3 9

10 Xen design consideration 5 level paging support for HVM expose LA57 feature to HVM extend guest page table traversing logic 5 level EPT in hypervisor introduce toolstack configurable CPUID.MAXPHYADDR 5 level EPT when CPUID.MAXPHYADDR > 48 4 level EPT otherwise 10

11 Xen design consideration 5 level paging support in Xen Difficulties: idle page table is constructed at build time, with precompile definitions, e.g. XEN_VIRT_START; PV dom0 shares same LA space with Xen; 5 level paging support for PV domain is still controversial(considering the arrival of PVH); Other opens: shadow support etc. Proposal: provide a basic enabling to boot Xen in 5 level paging mode; add new enhancement step by step(for some feature, enable when demanded). 11

12 Xen design consideration 5 level paging support in Xen Basic enabling: provide logic to support new paging mode with a separate binary; fabricate a 4 level paging mode to PV dom0(yet in de facto 5 level paging mode with hyperivsor); compile out CONFIG_SHADOW_PAGING at the first stage. Future enhancement: support impacted features such as IOMMU etc.; support runtime paging mode switch between 5 level and 4 level paging; support 5 level paging in shadow mode; whether or not to support 5 level paging in PV depends on the status of PVH dom0. 12

13 Q & A

Virtual Memory. CS 351: Systems Programming Michael Saelee

Virtual Memory. CS 351: Systems Programming Michael Saelee Virtual Memory CS 351: Systems Programming Michael Saelee registers cache (SRAM) main memory (DRAM) local hard disk drive (HDD/SSD) remote storage (networked drive / cloud) previously: SRAM