To REORG or not to REORG That is the Question. Kevin Baker BMC Software

Transcription

1 To REORG or not to REORG That is the Question Kevin Baker BMC Software

2 Objectives Identify I/O performance trends for DB pagesets Correlate reorganization benefits to I/O performance trends Understand the methods for collecting I/O performance data Identify object and application metrics that help identify objects in need of reorganization Establish a process for identifying application pagesets for analysis

3 Why Pageset Organization matters - the basics Its all about your data and the I/O required to access it. DB computes the best access path for an SQL statement to minimize I/O wait time. The access path chosen is primarily influenced by catalog statistics about the referenced tables and indexes. DB s primary mechanism for avoiding I/O wait time is asynchronous I/O via PREFETCH. PREFETCH is most effective in situations involving sequential processing, even if in small bursts, and is significantly affected by the physical ordering of the pages on disk.

4 About Access Paths - etch Given viable indexes, the DB Optimizer will try to use PREFETCH to read in the data pages asynchronously. If it can determine that sequential processing is reasonable when the access path is determined, it will call for SEQUENTIAL PREFETCH. If it identifies the need for a lot of specific records that are not sequentially located it may call for RID-based LIST PREFETCH. Even if the access path calls for random access (which involves synchronous I/O), runtime monitoring may invoke DYNAMIC PREFETCH if the pages requested appear to be even loosely sequential. 4

5 About Access Paths - BIND For STATIC SQL, the access path is determined at the time the program containing the SQL goes through the BIND process. This means that the table and index statistics in the catalog at the time of the BIND determine the access path, and it remains fixed until the next time a BIND is done. For DYNAMIC SQL, the access path is set by the PREPARE process every time the statement is executed (more or less). This means that the access path can change from run to run if the relevant catalog statistics are updated.

6 About Access Paths - Statistics There are several dozen statistics, kept in various catalog tables, that are used by the DB optimizer to select access paths. Many of these influence the choice of indexes, order of joins, etc. For the purpose of this discussion, we are interested in just a few that indicate organization level of the table or index. We will cover some of the statistics traditionally used to recommend REORGS later in the presentation. 6

7 About Access Paths - Statistics Tables (TABLESPACES/PARTITIONS) CARD (Cardinality) the number of rows in the tablespace or partition FARINDREF the number of rows relocated far from their original page Indexes (INDEXSPACES) CLUSTERRATIO the percentage of rows in clustering order LEAFFAR the percentage of leaf pages physically located far from the previous leaf page accessed in an index scan. 7

8 Impact on SQL Performance To explore the impact on SQL performance we set up some special tables, indexes, and SQL workloads. To minimize variables: We used test DB subsystems with stable configurations throughout the testing. We isolated measured pagesets to their own buffer pools that were large, and cleared prior to each run. We avoided workloads that would cause dis-organization to an extent that would drastically affect access path. Factors explored were inserts, updates that relocate rows, and free space. 8

9 Case : Dynamic Sequential after Updates DB version 8 k row table, no freespace, clustering index DYNAMIC SELECT workload; returns k rows access path using index and sequential prefetch UPDATE workload that updates k random rows in such a way that the rows have to be relocated. RUNSTATS for the table and index done after each update workload and key statistics captured. Access performance statistics for the SELECT workloads were gathered for both the table and index. 9

10 Case : Dynamic Sequential after Updates FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

11 Case : Dynamic Random after Updates DB version 8 k row table, no freespace DYNAMIC SELECT workload; returns k rows access path completely random UPDATE workload that updates k random rows in such a way that the rows have to be relocated. RUNSTATS for the table and index done after each update workload.

12 Case : Dynamic Random after Updates FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

13 Case : Dynamic Sequential after Inserts DB version 8 k row table, no freespace, clustering index DYNAMIC SELECT workload; returns k rows access path using index and sequential prefetch Insert workload that inserts random rows each run. RUNSTATS for the table and index done after each insert workload.

14 4 Case : Dynamic Sequential after Inserts FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

15 Case 4: Dynamic Random after Inserts DB version 8 k row table, no freespace DYNAMIC SELECT workload; returns k rows access path completely random Insert workload that inserts random rows each run. RUNSTATS for the table and index done after each Insert workload.

16 6 Case 4: Dynamic Random after Inserts 87 FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run 46 Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

17 Case : Dynamic Sequential after Updates without RUNSTATS DB version 8 k row table, no freespace, clustering index DYNAMIC SELECT workload; returns k rows access path using index and sequential prefetch UPDATE workload that updates k random rows in such a way that the rows have to be relocated. No RUNSTATS between runs so the catalog statistics are not being updated. 7

18 8 Case : Dynamic Sequential after Updates without RUNSTATS FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

19 Case 6: Static Sequential after Updates without Rebind or RUNSTATS DB version 8 k row table, no freespace, clustering index STATIC SELECT workload; returns k rows access path using index and sequential prefetch UPDATE workload that updates k random rows in such a way that the rows have to be relocated. No RUNSTATS and no REBIND between runs Final runs after REORG, and then after RUNSTATS and REBIND 9

20 Case 6: Static Sequential after Updates without Rebind or RUNSTATS After REORG, STATS, REBIND Final run after REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

21 Case 7: Dynamic Sequential after Updates, no RUNSTATS, with FREESPACE DB version 8 k row table, table PCTFREE = ; index PCTFREE = Clustering index Dynamic SELECT workload; returns k rows access path using index and sequential prefetch UPDATE workload that updates k random rows in such a way that the rows have to be relocated. No RUNSTATS between runs

22 Case 7: Dynamic Sequential after Updates, with FREESPACE FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

23 Chart of Case: Sequential after Updates without Freespace

24 4 Chart of Case7: Sequential after Updates with Freespace

25 Conclusions about SQL Performance and Pageset Organization Relocating update activity reduces organization, significantly affects sequential performance, can cause some increases in workload due to increased row sizes, Insert activity reduces organization, significantly affects sequential performance, can also increase workload (rows fetched) Random table access will not be significantly affected by reduced organization or improved by REORGs Index access can be degraded although this is usually a lesser impact Freespace delays worst performance impacts, at cost of unused disk space

26 Conclusions about SQL Performance and Pageset Organization Typical performance impacts include Increased getpage activity Can also be caused by increased workloads Increased sync I/Os, increased sync I/Os per getpage Can be masked by buffer pool tuning Updated statistics can help optimizer compensate Dynamic SQL, Static SQL if rebound Can also cause unexpected access path changes; could make things much worse RUNSTATS causes statement invalidation in the Dynamic statement cache. 6

27 Case 8: DB 9 - Dynamic Sequential after Updates, no FREESPACE DB version 9 k row table, no freespace, clustering index DYNAMIC SELECT workload; returns k rows access path using index and sequential prefetch UPDATE workload that updates k random rows in such a way that the rows have to be relocated. RUNSTATS for the table and index done after each update workload. 7

28 8 Case 8: DB 9 - Dynamic Sequential after Updates, no FREESPACE FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

29 Case 9: DB 9 - Dynamic Random after Updates, no FREESPACE DB version 9 k row table, no freespace DYNAMIC SELECT workload; returns k rows access path completely random UPDATE workload that updates k random rows in such a way that the rows have to be relocated. RUNSTATS for the table and index done after each update workload. 9

30 Case 9: DB 9 - Dynamic Random after Updates, no FREESPACE FINAL RUN REORG Run Run Run Run Run Run Run Run Run Run Initial Run pgs req FAR ratio DREF pgs req GP GP/ LEAF CLUST FARIN- CARD GP GP/ Thread Statistics (indexspace) Runstats Statistics Thread Statistics (tablespace)

31 DB 9 and Real-Time Statistics (RTS) DB version 9 has added the ability to dynamically maintain pageset statistics in separate catalog tables. SYSIBM.SYSTABLESPACESTATS SYSIBM.SYSINDEXSPACESTATS These values are maintained without RUNSTATS The Optimizer does not use them for access path analysis Many of the statistics are relative to the last REORG With an understanding of when they get updated, they are basically always available. These are available in v8 too. (even v7!) You just have to do some work to set them up.

32 Some RTS Statistics The RTS tables have the basic statistics we have been looking at: SYSIBM.SYSTABLESPACESTATS REORGLASTTIME timestamp of last REORG REORGNEARINDREF number rows relocated since REORG but near the original page REORGFARINDREF - number rows relocated since REORG far from the original page SYSIBM.SYSINDEXSPACESTATS NLEVELS - number of index levels REORGLEAFNEAR number of leaf pages relocated but near its previous logical leaf page REORGLEAFFAR number of leaf pages relocated far from its previous logical leaf page

33 Methods for deciding when to REORG Typical methods include Automatically on fixed schedule When certain catalog statistics breach a threshold Change in cardinality and degraded cluster ratio Degraded page ordering (FARINDREF) Degraded leaf page distribution, ordering, levels We can save resources used for REORGs if we Correlate those statistics with performance data Only REORG if and when needed

34 Collecting Performance Data DB IFCID 99 contains key statistics for each pageset with at least I/O per second average in a stat cycle: DBID, PSID, partition Getpage counts Sync I/Os Async I/Os Async pages read If activated, these records are produced on the regular DB statistics cycle. 4

35 Collecting Performance Data A custom program, SAS procedure, or vendor tool can be used to collect these records periodically (e.g. daily), summarize the numbers by pageset, add them to a performance statistics table, This performance statistics table can have columns for date-time, DBID, PSID, Partition, getpages, sync I/Os, async I/Os, async pages read. It s also necessary to know when REORGS are done add a column to indicate a REORG event

36 Real-Time Statistics With the availability of the RTS tables, it is now possible to capture key organization statistics, on a daily basis, easily correlate with performance statistics. Could capture them at the same time the performance statistics are summarized for the day. Could keep them in the same table. Solves the problem of capturing last REORG time. Now possible to do percentage change calculations on catalog statistics as well as performance statistics. 6

37 Collecting Performance Data - sample table layout CREATE TABLE PAGESET_PERFORMANCE_TABLE (COLLECT_TIME TIMESTAMP, DBID SMALLINT, PSID SMALLINT, PART SMALLINT, BPID SMALLINT, DBNAME CHAR(8), PSNAME CHAR(8), TYPE CHAR(), REORGLASTTIME TIMESTAMP, TABLE_FARINDREF INTEGER, TABLE_NEARRINDREF INTEGER, TABLE_REORGUNCLUSTINS INTEGER, TABLE_TOTALROWS INTEGER, INDEX_REORGLEAFFAR INTEGER, INDEX_REORGLEAFNEAR INTEGER, INDEX_NLEAF INTEGER, INDEX_TOTALENTRIES INTEGER, GETPAGES INTEGER, SYNC_IO INTEGER, GPPER INTEGER, ASYNC_IO INTEGER, ASYNC_PAGES INTEGER); 7

38 Developing REORG Triggers Performance data analysis can be used either to recommend REORGS as needed, or to study the results of REORGS and use the information to adjust fixed schedules. Either way, the analysis usually depends on a trigger, which is a metric or formula threshold that is used to decide whether a REORG is needed or not. The threshold part of these triggers often have to be tailored to the needs of each application. Note that physical state and storage use triggers, such as extents, percentage of dropped table rows, etc. are indicators that represent issues unrelated to performance. 8

39 Developing REORG Triggers Recommendations from the Administration Guide Table spaces, REORG if More than % of rows relocated far If clustering index, then CLUSTERRATIO< 9% Else number referenced rows far from optimal > % Index spaces, REORG if More than % of active leaf pages are far from optimal position The average distance between consecutive leaf pages exceeds More than a designated percentage of rows have been inserted or deleted 9

40 Developing REORG Triggers Factoring in degraded performance Tables paces or index spaces, REORG if Baseline prefetch pages is > x sync I/Os And sync I/Os have increased % since baseline And getpages per sync I/O have fallen % Any pageset nominated for REORG by the performance triggers and the catalog statistics triggers is a good candidate for REORG. 4

41 Developing REORG Triggers Post REORG analysis If using regularly scheduled REORGS, analysis of the performance data a day after the reorganization can indicate the degree of improvement. Interesting data points: (getpages, sync I/O, async pages, and getpages per sync I/O) Before the REORG, % increase in metrics since the last REORG (baseline). After the latest REORG, % decrease in metrics. Small values indicate REORG was too soon or not needed at all. Large values mean the REORG was too late. 4

42 Putting it all Together Activate Real-Time Statistics in DB (pre v9, do setup work) Activate DB statistics class 8 and begin recording IFCID 99 data Create the pageset performance table. Setup daily job to summarize 99 data, collect RTS data, and populate the pageset performance table. Either use the data to adjust REORG schedules, directly trigger REORGS, or perform post REORG analysis on benefits. 4

43 Conclusion With a little work it is possible to setup a process to capture pageset performance statistics and real-time object statistics. With this data better triggers can be developed that only recommend REORGS when performance has been degraded. Post REORG analysis of this data can help to refine trigger thresholds or adjust schedules to balance performance versus costs. 4

44 44 Kevin Baker BMC Software, Inc.