Taking Advantage of Oracle Rdb Memory Management Features. Norman Lastovica Oracle Rdb Engineering November 14, 06

Transcription

1 Taking Advantage of Oracle Rdb Memory Management Features Norman Lastovica Oracle Rdb Engineering November 14, 06

2 Overview Background Virtual Memory Translation Buffer CPU caches Alignment faults Resident Images Reserved Memory Registry Performance Impacts 2

3 Physical vs Virtual Memory Physical memory The memory chips in computer Virtual memory Mapped from virtual to physical Process s view of memory Protection / Permission Virtual can often be larger than physical 3

4 VMS Virtual address space 32-Bit addressing P0 1gb process private P1 1gb process private S0S1 2gb system-wide shared space 64-bit addressing 8 TB minimum P2 Process private S2 System-wide shared space 4

5 Page Unit of memory access protection VAX = 512 bytes Alpha & I64 currently = 8192 bytes Physical page sometimes called Page Frame 5

6 6 Physical & Virtual

7 Page Table Entry Describes virtual page Physical page location Per-mode page state, protection & permission 7

8 Page Table Collection of page table entries Private page tables Shared page tables for shared memory Shared page tables for system address space 8

9 Translation Buffer CPU cache of information from page table entry Limited number available EV7 has 128 ITB & 128 DTB Itanium has ITB and DTB Invalidated at certain events (i.e. page fault) 9

10 Alpha EV6/EV7 TB A demand-paged memory-management unit with translation buffer, which, when used with PALcode, can implement a variety of page table structures and translation algorithms. The unit consists of a 128-entry, fullyassociative data translation buffer (DTB) and a 128-entry, fully-associative instruction translation buffer (ITB), with each entry able to map a single 8KB page or a group of 8, 64, or 512 8KB pages. The allocation scheme for the ITB and DTB is round-robin. The size of each translation buffer entry s group is specified by hint bits stored in the entry. The DTB and ITB implement 8-bit address space numbers (ASN), MAX_ASN=

11 Granularity Hint (aka Huge Page) Single entry describes multiple pages Contiguous physical pages Contiguous virtual pages All have same protection characteristics Reduces translation buffer cache misses Varied size Alpha 8K, 64K, 512K, 4M I64 4K, 8K, 16K, 64K, 256K, 1M, 4M, 16M, 64M, 256M, 4G 11

12 Resident Images Image content resident in physical memory No page faults Granularity hints Mapped in system address space S0S1 S2 starting with I64 V8.3 Users share one copy of pages & page tables 12

13 Resident Global Sections Content resident in physical memory Mapped in P0, P1 or P2 address space No page faults Optional shared page tables All users share one copy of pages Saves memory for large sections & many users 13

14 CPU Cycle vs. Memory Latency rx4640 (1.6GHz) GS1280 (1.3Ghz) 0.63 nsec cycle time 0.77 nsec cycle time GS320 (32P/32C, 1.2GHz) Superdome (32P/32C, 1.5GHz) GS1280 (32P/32C, 1.3GHz) Arches/Montecito (16P/32C, 1.6 GHz) 825 nsec latency 417 nsec latency 225 nsec latency 334 nsec latency 14

15 CPU Caches Faster, smaller memory closer to CPU Copy of memory content for faster access Several levels of different sizes & speeds CPU 4kb L1 Cache 2mb L2 Cache 16gb Main Memory 15

16 Pre-Fetching into CPU Cache When needed memory known ahead of time Significant performance improvements possible for some algorithms Accessing all data in a buffer Can pre-fetch cache lines ahead of loop to reduce effective memory latency 16

17 Aligned & Unaligned Data expected to be on natural boundary If data known at compile-time to be unaligned, additional instructions generated If data known aligned at compile-time but is not aligned, hardware raises alignment fault & software fixes up reference 17

18 Reserved Memory Registry Optional list of pre-reserved physical memory at system boot for global sections Allows application global sections to be resident with shared page tables & granularity hints 18

19 Global Sections & Granularity Hint Regions Wells TNA27:> MCR SYSMAN RESERVED_MEMORY ADD NJL$SHARED_MEMORY /PAGE_TABLES /SIZE=1100 /ALLOCATE Wells TNA3:> SHOW MEMORY /RESERVE Memory Reservations (pages): Group Reserved In Use Type NJL$SHARED_MEMORY SYSGBL Page Table NJL$SHARED_MEMORY SYSGBL Allocated NJL$SHARED_MEMORY SYSGBL Allocated NJL$SHARED_MEMORY SYSGBL Allocated Total (1.07 GBytes reserved)

20 Using GH Regions 05: : : : : : : : : :43 03:23 CPU Time Single User 1Gb global section 100,000,000 loops Increment random QW rx p No GH rx p GH 20

21 Resident Images With Rdb Various Rdb images installed resident Pass /RESIDENT as parameter Execute prior to end of SYSTARTUP_VMS 21

22 Reserved Memory Resident Sections With Rdb Various Rdb global sections can be resident SHARED MEMORY IS PROCESS RESIDENT RMU/DUMP/HEADER to find section names & sizes SYSMAN to register the sections 22

23 Reserving Memory for Rdb Global Sections Nhgal8 TNA4:> SQL$ ALTER DATA FILE X$ - SHARED MEMORY IS PROCESS RESIDENT Nhgal8 TNA4:> RMU /DUMP /HEADER /OUTPUT=X.X X$ Nhgal8 TNA4:> SEARCH X.X SECTION NAME IS /WINDOW=10 - Global section size With global buffers disabled is 2,047,042 bytes (2MB) With global buffers enabled is 33,860,114 bytes (33MB) - Global Section Name is RDM72N$1$DGA C Row cache I7 Shared Memory... - Shared memory will be mapped resident - Global Section Name is RDM72R$1$DGA C Shared memory section requirement is 96,340,608 bytes (97MB) Nhgal8 TNA4:> MCR SYSMAN RESERVED_MEMORY ADD /ALLOCATE /SIZE=33 RDM72N$1$DGA C Nhgal8 TNA4:> MCR SYSMAN RESERVE_MEMORY ADD /ALLOCATE /SIZE=97 RDM72R$1$DGA C

24 Testing Effects of Features Application does one index-only lookup per transaction 100% CPU bound ~75% executive mode 4 users, no waiting 24

25 Rx ghz Baseline +Rdb Installed /RESIDENT +Registered RC & TROOT Sections 25

26 GS mhz Baseline +Rdb Installed /RESIDENT +Registered RC & TROOT Sections 26

27 Buffer Objects & Fast IO Avoid locking & probing buffer for each IO Enables use of FAST IO services (vs $QIO) Requires OpenVMS rights identifier RMU /SET BUFFER_OBJECT /ENABLE = 27

28 Buffer Objects & FAST IO Creating Indexes 43: : : : : : : :40.3 Elapsed CPU 14: : : Block Buffers BufferObjects 28

29 Credits & Special Thanks Joe Famularo John King Kevin St George Tom Beaudin Brian Allison Mark DeYoung Steve Lieman Martin Ramshaw Guy Peleg Roxanne Young Paul Mead Ian Smith Christian Moser Karen Noel Guenther Froehlin Greg Jordan Tom Cafarella John Reagan 29

30 Q & A A N S W E R S Q U E S T I O N S A N S W E R S norman.lastovica@oracle.com 30