DBMS Performance Tuning

Transcription

1 DBMS Performance Tuning

2 DBMS Architecture GCF SCF PSF OPF QEF RDF QSF ADF SXF GWF DMF Shared Memory locks log buffers Recovery Server Work Areas Databases log file

3 DBMS Servers Manages client access to databases Comprises functionality specific facilities By default one per installation Multi-threaded process Has system threads and user threads Configurable

4 Agenda Disk Location layout CPU Threads Memory Caches

5 Disk Locations Tables Pages

6 Locations Names by which Ingres points to a disk area At installation default locations are set up for Database Journal Dump Checkpoint Work Use locations to spread disk IO

7 Locations A database can be extended to use extra database and work locations A table Can be spread across locations Spread across too many locations may degrade performance Need to understand disk set up for WAN SAN Raid

8 Locations Alternative locations can be created through Accessdb SQL statements Extenddb To use a location a table must be Relocated Reorganized System catalogs are always in the database home location

9 Tables Single location verses multiple locations Page size Compressed verses non-compressed Indexing Table structure

10 Partitioned Tables Queries can scan relevant partitions Can make use of Parallel Query Table partitions can be in different locations A partition can be in multiple locations A table can be partitioned by a distribution HASH List Range Automatic

11 Example Create table employee ( emp_no integer not null not default, emp_name char(32) not null not default, dept_no integer) with partition = ( (list on dept_no partition p1 values (1, 2), partition p2 values (3, 4), partition p3 values (DEFAULT)) subpartition (hash on emp_no 16 partitions) ); modify employee to btree unique on dept_no, emp_no;

12 Pages Page size = DMF cache size used Functionality verses performance Lock level Table Page Row

13 CPU User threads Internal Threads Other things

14 User threads Number of concurrent connections SQL sent Data returned

15 Internal threads Dead Process Detector thread Force Abort thread Group Commit thread Consistency Point(CP) thread Event thread WriteBehind(WB) thread (for each configured cache) Replicator Queue Manager thread Terminator thread Parallel Query threads Parallel Sort threads Log writer threads

16 Parallel Query Partitioning not required for parallel query Partitioning enables more parallelism Partition compatible joins in parallel threads

17 Parallel Query opf_pq_dop Defines the maximum number of exchange nodes that can be compiled into a query plan 0 or 1 prevents the generation of parallel plans Any other positive value enables them Default 8 Maximum 32

18 Parallel Query opf_pq_partthread Undocumented and not in config.dat Defines the maximum number of threads per exchange node Default 2 Testing show that more than 6 adds no value

19 Parallel Query opf_pq_threshold Defines the cost threshold that a query plan must exceed before a parallel plan is generated The cost estimate is the sum of the C and D numbers at the top of the query plan Default 1000

20 Parallel Query Set [No]Parallel <nn> Controls the generation of parallel query plans Can be used to override opf_pq_dop nn indicates the degree of parallelism The number of exchange nodes in a plan Default 4 Maximum 32

21 Parallel Sort psort_rows Sets the threshold in rows for doing a parallel sort Sorts of fewer than psort_rows rows will not consider a parallel sort The default is rows

22 Parallel Sort psort_nthreads Controls the number of threads used when parallel sorting The default is 2 threads A machine with many CPU's and large sorts may benefit from more worker threads Too many threads Will not help sorting May be slower due to contention

23 Log Writer threads 1 is not enough Start with 4 or 5 Monitor wait-for-logwriter in logstat Minimize wait-for-logwriter Check log disk if more than 20 configured Consider change to log buffer size

24 Memory On 32 bit maximum of 4 gigabytes On 64 bit lots more available DBMS Workspace DBMS Resource Logging Locking

25 DBMS Workspace For example: opf_memory psf_memory qef_qep_mem rdf_memory stack_size Memory Allocated in chunks up to the workspace limit Not obvious how much a session is using

26 DBMS Workspace For large installations Default configuration creates large workspaces Larger does not mean faster Smaller does not mean less resource usage If queries work Do not change Workspace Parameters If some queries take too long Look at the SQL before changing Parameters, tables structures or indexes

27 DBMS Resource For example: dmf parameters qsf_memory qef_sort_mem rdf_max_tbls For large installations Default configuration may set parameters too small Larger may mean faster Smaller may mean flushing

28 DMF Defaults are too small Minimum of 1000 pages for single page caches Larger cache may give gains Massive caches may have decreasing benefit Increase and test 2006 is better than previous releases Tune intelligently - use the tools IMA DM420 Monitor Cache Statistics

29 DMF: Single page buffers Tune caches for Users Transaction throughput Table usage Allow enough pages so that FHDR's, FMAP's, btree root and index pages do not get flushed Consider using a separate cache for session tables Session tables can be large Session tables can be numerous Session tables do get written to disk if the DMF cache is required

30 DMF: CBF - dbms.cache.8k.derived Derived DBMS Cache Parameters for 8k Name Value Units Protected dmf_cache_size data pages yes dmf_free_limit 1312 data pages no dmf_group_count 4500 group buffers no dmf_memory bytes no dmf_modify_limit data pages no dmf_wb_end data pages no Tuned cache in stages: default was inadaquate

31 DMF: Single page buffers : DM420 before =====Thu Aug 2 04:37: ==========! Buffer Cache Configuration ( 8K) ! 8 Buffer count:78000 Bucket count:77999 Group count:4500 Size:! Free count: 8099 Limit:498 Modify count:7911 Limit:13300! Free group count: 4875 Modify group count: 125! Fixed count: 0 Group fixed count: 0! WB start limit: 7500 WB end limit: 5500! WB flushes: 324, Agents cloned: 433! Agents active: 0, Agent hwm: 6! WB pages flushed: 0, WB groups flushed: 0! Summary Statistics ( 8K) ! GWAIT GSYNC FREEWAIT GW PAGES GW I/O! ! FIX CALLS HITS CHECK REFRESH READ TOSS!

32 DMF: Single page buffers : DM420 after =====Thu Aug 2 04:37: ==========! Buffer Cache Configuration ( 8K) ! 8 Buffer count:78000 Bucket count:77999 Group count:4500 Size:! Free count: Limit:1312 Modify count:3249 Limit:31500! Free group count: 4286 Modify group count: 214! Fixed count: 0 Group fixed count: 0! WB start limit: WB end limit: 12600! WB flushes: 120, Agents cloned: 334! Agents active: 0, Agent hwm: 6! WB pages flushed: 0, WB groups flushed: 0! Summary Statistics ( 8K) ! GWAIT GSYNC FREEWAIT GW PAGES GW I/O! ! FIX CALLS HITS CHECK REFRESH READ TOSS!

33 DMF: Single page buffers : DM420 before Stats for FHDR pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for FMAP pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for ROOT pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for INDEX pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for LEAF pages:! FIX CALLS HITS CHECK REFRESH READ TOSS!

34 DMF: Single page buffers : DM420 after Stats for FHDR pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for FMAP pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for ROOT pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for INDEX pages:! FIX CALLS HITS CHECK REFRESH READ TOSS! snip! Stats for LEAF pages:! FIX CALLS HITS CHECK REFRESH READ TOSS!

35 DMF: Group buffers Tune buffers for queries Tune queries to keep group buffer usage to a minimum Switch on Gather Write Setting Group Buffers to zero does not result in no group buffers

36 QSF Query Storage Facility Global memory area Used for storing query objects Used to pass information between facilities Monitor using Trace points IMA QS501 QS512

37 Logging Log buffers Number Size logstat statistics to tune log parameters Log Waits: Free Buffer Split Buffer Buffer utilization profile All logwriters busy Max write queue length Max write queue count

38 Logging Log buffers Number - default 35 (recom. 200) Size - default 4K (recom. 8K) logstat statistics to tune log parameters Log Waits: Free Buffer Split Buffer Buffer utilization profile All logwriters busy Max write queue length Max write queue count

39 Logging Log buffers Number - default 35 (recom. 200) Size - default 4K (recom. 8K) logstat statistics to tune log parameters Log Waits: Free Buffer Split Buffer Buffer utilization profile All logwriters busy Max write queue length < Number of buffers Max write queue count

40 Logging ----Current log file header Block size: 8192 Block count: Partitions: 1 Buffer count: 200 CP interval: 5120 Logfull interval: Abort interval: Last Transaction Id: B4B0FBE8F Begin: < :37587:2448> CP: < :42735:5956> End: < :43170:1224> Percentage of log file in use or reserved: 8 Log file blocks reserved by recovery system: 0 Archive Window: < ,42725,5956>..< ,43170,1224> Previous CP: < ,37587,2448> Status: ONLINE,ECPDONE Active Log(s): II_LOG_FILE

41 Logging ====Fri May 11 01:42: Logging System Summary====== Database add 1068 Database removes 1014 Transaction begins (6M) Transaction ends Log read i/o's Log write i/o's Log writes Log forces 2807 Log waits Log split waits 0 Log free waits 0 Log stall waits 8 Log BCP waits 8 Logfull stall waits 0 Log group commit 8337 Log group count 8379 Check commit timer 2326 Timer write 2326 Timer write, time 2 Timer write, idle 2324 Inconsistent db 0 Kbytes written ii_log_file read ii_dual_log read 0 write complete dual write complete 0 All logwriters busy 1369 Max write queue len 85 Max write queue cnt Buffer utilization profile <10% ********* 10-19% *************** 20-29% ********************** 30-39% ** 40-49% *** 50-59% ** 60-69% * 70-79% * 80-89% * >90% *********

42 Logging ====Thu Aug 2 04:37: Logging System Summary====== Database add 7558 Database removes 7396 Transaction begins (23M) Transaction ends Log read i/o's Log write i/o's Log writes Log forces Log waits Log split waits 0 Log free waits 0 Log stall waits 32 Log BCP waits 32 Logfull stall waits 0 Log group commit Log group count Check commit timer 7782 Timer write 7782 Timer write, time 0 Timer write, idle 7782 Inconsistent db 0 Kbytes written ii_log_file read ii_dual_log read 0 write complete dual write complete 0 All logwriters busy 1001 Max write queue len 99 Max write queue cnt Buffer utilization profile <10% ************* 10-19% ***************** 20-29% *************** 30-39% ***** 40-49% **** 50-59% ** 60-69% * 70-79% * 80-89% * >90% *******

43 Locking Large installations can generate huge Lock limits Resource limits Result in large lock/log shared memory segments lockstat statistics to tune lock parameters High-water marks for: Lock lists Locks Resources Max locks per transaction

44 Locking =====Fri May 11 01:42: Locking System Summary================= Total Locks Total Resources Locks per transaction 3000 Lock hash table Locks in use 956 Resource hash table Resources in use 902 Total lock lists 530 Lock lists in use 83 =====Fri May 11 01:42: Locking System Statistics============== Create lock list Release lock list Request lock Re-request lock Convert lock Release lock Escalate 467 Lock wait 80 Convert wait 0 Convert Deadlock 0 Deadlock Wakeups 21 Max dlk queue len 1 Deadlock Search 21 Deadlock 0 Cancel 0 Convert Search 0 Allocate CB Deallocate CB LBK Highwater 3 LLB Highwater 95 SBK Highwater 3 LKB Highwater 6019 RBK Highwater 9 RSB Highwater 5975 Max Local dlk srch 0 Dlk locks examined 21 Max rsrc chain len 22 Max lock chain len 21 Max locks per txn 3001 Callback Wakeups 0 Callbacks Invoked 0 Callbacks Ignored 0

45 Locking =====Thu Aug 2 04:37: Locking System Summary================= Total Locks Total Resources Locks per transaction Lock hash table Locks in use 1047 Resource hash table Resources in use 968 Total lock lists 2160 Lock lists in use 102 =====Thu Aug 2 04:37: Locking System Statistics============== Create lock list Release lock list Request lock Re-request lock Convert lock Release lock Escalate 652 Lock wait 196 Convert wait 5 Convert Deadlock 0 Deadlock Wakeups 122 Max dlk queue len 4 Deadlock Search 130 Deadlock 1 Cancel 1 Convert Search 0 Allocate CB Deallocate CB LBK Highwater 3 LLB Highwater 119 SBK Highwater 3 LKB Highwater RBK Highwater 10 RSB Highwater Max Local dlk srch 2 Dlk locks examined 150 Max rsrc chain len 48 Max lock chain len 47 Max locks per txn 5175 Callback Wakeups 0 Callbacks Invoked 0 Callbacks Ignored 0

46 A few words about tuning Have a plan Tune with care Know the performance baseline Change one thing at a time Measure the change Understand parameter interactions Monitor important E_* E_DMA00D_TOO_MANY_LOG_LOCKS

47 A few words about tuning Have a plan Tune with care Know the performance baseline Change one thing at a time Measure the change Understand parameter interactions Monitor important E_* errors. E_DMA00D_TOO_MANY_LOG_LOCKS E_DM004B_LOCK_QUOTA_EXCEEDED E_DM9041_LK_ESCALATE_TABLEMaximum number of locks for transaction exceeded. Escalating to table-level lock on table iiprotect in database mydb

48 A few more words about tuning Tuning is not difficult Monitoring is a continuous process Be pro-active Health checks can save on Hardware costs Non-stressed installations run for longer

49 Summary Tune with care Have a plan Go on a training course Ask an expert

50 Questions & Answers

51 DBMS Performance Tuning