Exploring Memory in DB2 9.5 and Beyond

Size: px

Start display at page:

Download "Exploring Memory in DB2 9.5 and Beyond"

Arline Stewart
6 years ago
Views:

1 Exploring Memory in DB2 9.5 and Beyond Michael Cornish, Senior Software Analyst Advanced Problem Diagnostics Team IBM Toronto Lab 2009 IBM Corporation

2 TOPICS Memory Architecture Changes for DB2 9.5/9.7 Memory Model Update Instance Memory STMM IBM Corporation

3 Memory Architecture Changes Why? Reduce knobs which must be configured No need to configure for memory which is absolute requirement Database Heap, Application Control Heap, Application Heap Focus on tuning performance-related memory areas (eg. bufferpools) Make memory allocation more dynamic Avoid recycling, Increase STMM tuning capability Made possible by: Moving away from 32-bit constraints (virtual process address space) Multithreaded model (reduce need for fixed shared memory areas) IBM Corporation

4 Memory Model Update DB2 9.5 Model FCM MEMORY (shared across all local partitions) ( portion billed to this partition s Instance Memory ) INSTANCE MEMORY (per partition) DBMS memory FMP memory db2start database activation Database A Memory APPL Memory Private Memory Database B Memory APPL Memory LOCAL LOCAL Communication Memory Communication Memory per local connection db2start IBM Corporation

5 Memory Model Update DB2 9.5 Model - DPF FCM MEMORY (shared across all local partitions) INSTANCE MEMORY (Partition 0) DBMS FMP INSTANCE MEMORY (Partition 1) DBMS FMP Database A Memory APPL Database B Memory APPL Database A Memory APPL Database B Memory APPL Private Memory Private Memory LOCAL LOCAL LOCAL LOCAL DPF 2 partitions IBM Corporation

6 Memory Model Update How to see it in DB2? db2pd dbptnmem (sysproc.admin_get_dbp_mem_usage) High-level, all areas, for monitoring Instance Memory usage includes allowance for configuration, not just actual system memory usage Mem Used includes Cached Individual Memory Consumers: Name Mem Used (KB) HWM Used (KB) Cached (KB) ======================================================== APPL-SAMPLE DBMS-mcornish FMP_RESOURCES PRIVATE LCL-p DB-SAMPLE FCM_RESOURCES IBM Corporation

7 Memory Model Update How to see it in DB2? Shared Memory: db2pd inst memsets db <database> -memsets Commited (Cmt) value is accurate measure of system memory usage More detailed, useful, no SQL Can map to OS resources (IPC shared segment IDs) Memory Set Name Id Size(Kb) Cmt(Kb) SAMPLE (DB/Database Memory) AppCtl (APPL Memory) App (Local Communication) DBMS FMP FCM IBM Corporation

8 Memory Model Update How to see it in DB2? Private memory: db2 get snapshot for database manager Committed private Memory (Bytes) = db2pd dbptnmem (Used Cached) Name Mem Used (KB) HWM Used (KB) Cached (KB) ======================================================== PRIVATE Greater reliance on operating system tools as private memory includes more than just DB2 s tracked allocations - ps, pmap, etc IBM Corporation

9 Memory Model Update How to see it in OS? Shared ipcs a (UNIX) * virtual size only * pmap (Solaris, Linux) svmon (AIX) Private ps (except Solaris, Linux) pmap (Solaris, Linux) Other glance (HP) perfmon (Windows) IBM Corporation

10 Memory Model Update Private Memory FCM MEMORY (shared across all local partitions) ( portion billed to this partition s Instance Memory ) INSTANCE MEMORY (per partition) DBMS memory FMP memory Database A Memory APPL Memory Private Memory Database B Memory APPL Memory LOCAL LOCAL Communication Memory Communication Memory db2start IBM Corporation

11 Memory Model Update Private Memory Single large memory area per partition, allocated as needed At OS level, shared by all DB2 threads (EDUs/Agents) At DB2 level, most memory is private to a thread, i.e. DB2 manages memory in thread-specific memory pools Non-configurable except private sort ( SHEAPTHRES > 0 ) Per-thread memory footprint is MB IBM Corporation

12 Memory Model Update Private Memory Consolidate, simplify Private Memory, eliminate bloat Active Agent DB2 9.1 DB2 9.5 Active Agent Active Agent Statement Heap Application Heap Statistics Heap APPLICATION MEMORY (APPL_MEMORY) Unused Commited Inactive Agent Cached Unused Private Memory Inactive Agent Agent/ Thread 1 PRIVATE MEMORY Agent/ Thread 2 Agent/ Thread IBM Corporation

13 Memory Model Update Database Memory FCM MEMORY (shared across all local partitions) INSTANCE MEMORY (per partition) DBMS memory FMP memory database activation Database A Memory APPL Memory Private Memory Database B Memory APPL Memory LOCAL LOCAL Communication Memory Communication Memory IBM Corporation

14 Memory Model Update Database Memory One Database Memory area per database, per partition Configurable via DATABASE_MEMORY SETTINGS: <value>, AUTOMATIC, COMPUTED AUTOMATIC, COMPUTED same behaviour with STMM OFF Size based on bufferpools, dbheap, utility heap, etc. AUTOMATIC allows STMM tuning if STMM ON **AUTOMATIC now available on ALL UNIX** **ONLY memory area tuned by STMM** IBM Corporation

15 DB2 9.5 / Linux AUTOMATIC Restriction Can t set both Database Memory and Instance Memory to AUTOMATIC Applies to Linux levels before SLES 10 SP1, RHEL 5 Lacking good system statistics, can t rely on STMM to keep DB2 s memory usage within system capacity Doesn t affect DB2 9.7 due to higher Linux level prereq Need one of: DATABASE_MEMORY = COMPUTED or <value> INSTANCE_MEMORY = <value> IBM Corporation

16 Memory Model Update Database Memory OVERFLOW Package Cache Package Cache Catalog Cache Utility Heap Database Heap ~20% Overflow Reserved LOCKLIST Shared Sortheap BUFFERPOOLS DATABASE MEMORY IBM Corporation

17 Memory Model Update Application Memory FCM MEMORY (shared across all local partitions) ( portion billed to this partition s Instance Memory ) INSTANCE MEMORY (per partition) DBMS memory FMP memory database activation Database A Memory APPL Memory Private Memory Database B Memory APPL Memory LOCAL LOCAL Communication Memory Communication Memory IBM Corporation

18 Memory Model Update Application Memory One Application Memory area per database, per partition Configurable via APPL_MEMORY Options: Specific Value, AUTOMATIC Should not need to be tuned (leave at AUTOMATIC) Volatile memory area which is efficiently self-managed Most memory is a strict requirement as opposed to performance/cache Memory usage dependent on number of connections, activity IBM Corporation

19 Memory Model Update Application Memory Statistics Heap STAT_HEAP_SZ Application Group Shared Heap (persistent,non-configurable) APPLICATION MEMORY (APPL_MEMORY) Statement Heap STMTHEAP Statement Heap STMTHEAP Statement Heap STMTHEAP Application Heap APPLHEAPSZ Application Heap APPLHEAPSZ Application Heap APPLHEAPSZ.. All Memory Heaps/Pools grow as needed, release memory when no longer used IBM Corporation

20 Memory Model Update Application Memory No more Application Groups (DPF, Intra-parallel, Concentrator) No more complex configuration, memory bloat DB2 9.1 DB2 9.5 APP_CTL_HEAP_SZ =? GROUPHEAP_RATIO =? APPGROUP_MEM_SZ =? APPL_MEMORY = AUTOMATIC Appgroup 1 Appgroup 2 Appgroup 3 APPLICATION MEMORY Appgroup 4 Appgroup IBM Corporation

21 Memory Model Update DB2 9.5 Model Questions? FCM MEMORY (shared across all local partitions) ( portion billed to this partition s Instance Memory ) INSTANCE MEMORY (per partition) DBMS memory FMP memory db2start database activation Database A Memory APPL Memory Private Memory Database B Memory APPL Memory LOCAL LOCAL Communication Memory Communication Memory per local connection db2start IBM Corporation

22 Memory Architecture Changes - Dynamic Shared Memory STMM tuning of Database Memory on all UNIX already available in version 9.1 on AIX and Windows No need to configure many memory parameters No fixed shared memory size (remember 32-bit?) On increase, shared memory segment added On decrease, underlying system memory released Virtual process allocation/mapping is persistent IBM Corporation

23 Memory Architecture Changes - Dynamic Shared Memory New Allocation Requirements decrease Memory Requirements Increase Uncommitted Memory Committed Memory (Backed by system i.e. RAM) Committed Memory Committed Memory Decommitted memory Decommitted memory Committed Memory Decommit frees underlying system memory, virtual allocation remains IBM Corporation

24 Memory Architecture Changes - MORE AUTOMATIC! Database Manager Memory MON_HEAP_SZ (Monitor Heap) Database Memory DBHEAP (Database Heap) Application memory APPL_MEMORY itself (formerly APPGROUP_MEM_SZ) APPLHEAPSZ (Application Heap) - includes App Control Heap STAT_HEAP_SZ (Statistics Heap) STMTHEAP (Statement Heap) *no change on migration* IBM Corporation

25 Memory Architecture Changes - MORE AUTOMATIC! Meaning of AUTOMATIC for new settings: Unlimited growth (up to any higher-level limit) OK as these have strict requirements not for performance STMTHEAP: dynamic join limited -> greedy join unlimited Other meanings of AUTOMATIC STMM Automatically tune Database Memory areas Automatically calculated/computed setting FCM grow and shrink based on demand IBM Corporation

26 Memory Architecture Changes AGENT_STACK_SZ Stacks managed separately from main DB2 memory model Fixed stack size required for multithreaded model Allocated as needed (up to limit), released when agent terminates Overall stack usage is insignificant Only specific activity, eg. statement compilation, may require large stack If not large enough, SQL0973 returned, stack size must be increased (not dynamic) SQL0973 generally treated as product defect need to reduce stack usage IBM Corporation

27 BEST PRACTICES New Memory Settings APPL_MEMORY = AUTOMATIC APPLHEAPSZ = AUTOMATIC STAT_HEAP_SZ = AUTOMATIC **STMTHEAP = <value> Improved AUTOMATIC behaviour change coming DB2 9.5 Fix Pack (IC68794), DB2 9.7 Fix Pack 3 (IC69070) DATABASE_MEMORY = AUTOMATIC (other options) DBHEAP = AUTOMATIC (<value>) Value is reserved use default value i.e. AUTOMATIC (1200) for small database DBMS - Database Manager Memory (not configurable) MON_HEAP_SZ = AUTOMATIC (<value>) Value is reserved use default value i.e. 90 for small database AGENT_STACK_SZ Default 1024 for OLTP, 4096 for DSS/Warehouse, Mixed IBM Corporation

28 INSTANCE MEMORY What is it? FCM MEMORY (shared across all local partitions) ( portion billed to this partition s Instance Memory ) INSTANCE MEMORY (per partition) DBMS memory FMP memory db2start database activation Database A Memory APPL Memory Private Memory Database B Memory APPL Memory LOCAL LOCAL Communication Memory Communication Memory per local connection db2start IBM Corporation

29 Instance Memory What is it? New accounting method Tracks main DB2 memory areas (Database, Application ) Accounts for both usage and configuration (eg. reservations) Memory Manager Gives permission for memory areas to grow update db cfg using Database_Memory <value> IBM Corporation

30 Instance Memory What is it? db2pd dbptnmem Database Partition Memory Controller Statistics Controller Automatic: Y Memory Limit: KB Current usage: KB HWM usage: KB Cached memory: KB Individual Memory Consumers: Name Mem Used (KB) HWM Used (KB) Cached (KB) ======================================================== APPL-SAMPLE DBMS-mcornish FMP_RESOURCES PRIVATE FCM_RESOURCES DB-SAMPLE IBM Corporation

31 Instance Memory What is it? OVERFLOW (cached) Cached Package Cache Package Cache Utility Overflow (buffer) Reserved Used Catalog Cache Heap Database Heap LOCKLIST Shared Sortheap BUFFERPOOLS DATABASE MEMORY RESERVED IBM Corporation

32 Instance Memory What is it? Used = Current size (In use by pools/heaps + reserved for configuration + overflow) Cached = Reducible, Unused overflow (DB, DBMS, FCM) ( can be reduced to make room elsewhere or reduce memory usage ) Individual Memory Consumers: Name Mem Used (KB) HWM Used (KB) Cached (KB) ======================================================== APPL-SAMPLE DBMS-mcornish FMP_RESOURCES PRIVATE LCL-p DB-SAMPLE FCM_RESOURCES IBM Corporation

33 Instance Memory INSTANCE_MEMORY INSTANCE_MEMORY (Database Manager configuration) AUTOMATIC or <value> AUTOMATIC computed value = 75-95% of RAM / # local partitions limit enforcement removed in DB2 9.5 FP5, DB2 9.7 FP1 except license memory limit still enforced <value> - Limit is enforced monitoring/sizing of Instance Memory important - Provides a target for STMM (ignores OS) - DPF Limitation single value for all partitions IBM Corporation

34 Free Used (includes Cached/ Reserved) Instance Memory vs. System Memory committed Database Memory committed Application Memory INSTANCE_MEMORY = 48GB OS & other Apps Instance Memory (Uncommited) Instance Memory (Committed) 64GB RAM Instance Memory Usage Accounting Difference System Memory Usage committed Private Memory IBM Corporation

35 Instance Memory - Summary Provides an overall limit on DB2 memory consumption Some areas not included (stack, memory from other software layers) Greater control with multiple instances Ensure safety room (taken care of by STMM) Free = Limit Used + Cached = / = 35% db2pd -dbptnmem: Memory Limit: KB Current usage: KB HWM usage: KB Cached memory: KB IBM Corporation

36 STMM DB2 s Self-Tuning Memory Manager Tunes most Performance memory Bufferpools, Locklist, Sort, Package Cache Targets optimal memory consumption OS memory statistics, performance statistics Similar to Operating System behavior OS uses available memory for file cache STMM makes use of available memory if sufficient benefit Can tune multiple databases within and across instances STMM controllers for each database compare benefits Equilibrium is reached when each database is getting the same marginal benefit from memory IBM Corporation

37 STMM : DATABASE_MEMORY = AUTOMATIC INSTANCE_MEMORY = AUTOMATIC (no limit, STMM monitors OS) DATABASE Untouchable memory kernel, other Apps, minimum file MEMORY cache Free System Memory Target More Benefit STMM DATABASE MEMORY Less Benefit Sorts & Hash Joins DB2 memory usage (outside of Database Memory) IBM Corporation

38 STMM : DATABASE_MEMORY = AUTOMATIC INSTANCE_MEMORY = AUTOMATIC (no limit, STMM monitors OS) DATABASE MEMORY Untouchable memory kernel, other Apps, minimum file cache If increase, STMM compensates Free System Memory Target DATABASE MEMORY More Benefit STMM Less Benefit Sorts & Hash Joins If increase, STMM compensates DB2 memory usage (outside of Database Memory) IBM Corporation

39 STMM : DATABASE_MEMORY = AUTOMATIC DATABASE MEMORY INSTANCE_MEMORY = <value> (STMM monitors Instance Memory) Free Instance Memory Target More Benefit STMM DATABASE MEMORY Less Benefit Sorts & Hash Joins DB2 memory usage outside of Database Memory IBM Corporation

40 STMM Multiple Databases System Memory Note NO Instance Boundaries Kernel, Other Application Memory, Minimum File Cache STMM Database A Free System Memory Target STMM Database B STMM Database C STMM Database D IBM Corporation

41 STMM Multiple Databases System Memory Kernel, Other Application Memory, Minimum File Cache STMM Database A Increased Benefit Free System Memory Target STMM Database B Decrease Database C STMM STMM Database D IBM Corporation

42 STMM Multiple Databases Instance Memory INSTANCE_MEMORY = <value> STMM Database A INSTANCE MEMORY LIMIT Free Instance Memory Target STMM Database B STMM Database C STMM Database D IBM Corporation

43 STMM Multiple Databases - Hybrid Kernel, Other Application Memory, Minimum File Cache Instance 1 (unlimited) Database A STMM Instance 3 (limited) STMM Database C Free System Memory Target Free Instance Memory Target Instance 2 (unlimited) Database B STMM STMM Database D IBM Corporation

44 STMM Miscellaneous Configuration SELF_TUNING_MEM is the master switch Configuration parameters and buffer pool sizes should be set to AUTOMATIC to enable tuning Persistence of changes All configuration changes are written to disk. If you start and stop the database all changes are preserved. Logging All configuration changes are logged to db2diag.log. Internal tuning information is logged to STMM log. Best used with the STMM log parser IBM Corporation

45 BEST PRACTICES Summary AUTOMATIC, AUTOMATIC, AUTOMATIC Put Instance Memory Cap on Databases which may benefit from more memory but where performance is not important Rogue queries may penalize other Databases competing for same resources STMM treats all databases equal Extremely volatile databases may require some fixed values for Performance Memory STMM is reactive, major workload changes need time to achieve optimal performance IBM Corporation

46 Exploring Memory in DB2 9.5 and Beyond? IBM Corporation

Tuning DBM and DB Configuration Parameters

DB2 for Linux, UNIX, Windows Tuning DBM and DB Configuration Parameters Philip K. Gunning DB2 Master Practitioner Gunning Technology Solutions, LLC 21 September 2005 DB2 is a registered trademark of IBM