Cost Based Optimizer CBO: Configuration Roadmap

Size: px

Start display at page:

Download "Cost Based Optimizer CBO: Configuration Roadmap"

Jean Jacobs
5 years ago
Views:

1 Cost Based Optimizer CBO: Configuration Roadmap Christian Antognini Sandro Crepaldi DOAG Regionaltreffen Hamburg/Nord , Hamburg

Disclaimer > The CBO changes from release to release > It s difficult to give general advice about many releases > Therefore, this presentation focuses on the most important

2 Disclaimer > The CBO changes from release to release > It s difficult to give general advice about many releases > Therefore, this presentation focuses on the most important versions at present» Oracle9i Release 2» Oracle 10g Release 1 > When information applies to only one version, one of the following icons will be used CBO: A Configuration Roadmap

3 Correctly configuring the CBO (1) > At least since Oracle9i the CBO works well, i.e. it generates good execution plans for most SQL statements > This is only true when» It is correctly configured» The database has been designed to take advantage of all its features > Correctly configuring the CBO is not an easy job This is no good reason for not doing it however! CBO: A Configuration Roadmap

4 Correctly configuring the CBO (2) > There is no single configuration that is good for every system» Each application has its own requirements» Each system has its own characteristics > Therefore, I m not able to provide you with the magic configuration > I can only show you how I proceed to do such a configuration CBO: A Configuration Roadmap

5 Configuration = INIT.ORA parameters + statistics > Statistics» Object statistics» System statistics > Basic» OPTIMIZER_MODE» OPTIMIZER_FEATURES_ENABLE» OPTIMIZER_DYNAMIC_SAMPLING» OPTIMIZER_INDEX_COST_ADJ» OPTIMIZER_INDEX_CACHING > Query transformation» QUERY_REWRITE_ENABLED» QUERY_REWRITE_INTEGRITY» STAR_TRANSFORMATION_ENABLED > Memory» WORKAREA_SIZE_POLICY» PGA_AGGREGATE_TARGET» HASH_AREA_SIZE» SORT_AREA_SIZE» BITMAP_MERGE_AREA_SIZE > Miscellaneous» DB_FILE_MULTIBLOCK_READ_COUNT» CURSOR_SHARING» SKIP_UNUSABLE_INDEXES» PARALLEL_*» Many undocumented parameters CBO: A Configuration Roadmap

6 Set the right parameter! (1) > Each INIT.ORA parameter has been introduced for a specific reason! > Understanding how it changes the behavior of the CBO is essential > Don t tweak the configuration randomly, instead» Understand the current situation» Define the goal to be achieved» Find out which parameter should be changed to achieve the goal CBO: A Configuration Roadmap

7 Choose the right tool! CBO: A Configuration Roadmap

8 Set the right parameter! (2) OPTIMIZER_FEATURES_ENABLED DB_FILE_MULTIBLOCK_READ_COUNT system statistics OPTIMIZER_INDEX_COST_ADJ OPTIMIZER_INDEX_CACHING object statistics OPTIMIZER_DYNAMIC_SAMPLING PGA_AGGREGATE_TARGET WORKAREA_SIZE_POLICY BITMAP_MERGE_AREA_SIZE HASH_AREA_SIZE SORT_AREA_SIZE QUERY_REWRITE_ENABLED OPTIMIZER_MODE CBO: A Configuration Roadmap

9 Configuration roadmap Once set to their optimal value they should not be modified further; the only exception is when tweaking the index related parameters does not achieve good results! OPTIMIZER_MODE OPTIMIZER_FEATURES_ENABLE DB_FILE_MULTIBLOCK_READ_COUNT OPTIMIZER_DYNAMIC_SAMPLING gather system and object statistics PGA_AGGREGATE_TARGET WORKAREA_SIZE_POLICY auto manual HASH_AREA_SIZE SORT_AREA_SIZE BITMAP_MERGE_AREA_SIZE test the application most of the execution plans are good yes no gather missing histograms OPTIMIZER_INDEX_COST_ADJ OPTIMIZER_INDEX_CACHING CBO: A Configuration Roadmap

10 Object statistics > The object statistics describe the data stored in the database» Table statistics: number of rows, number of blocks below the high water mark and average row length» Column statistics: number of distinct values, number of NULL values and data distribution (a.k.a. histograms)» Index statistics: number of distinct keys, high, number of leaf blocks and clustering factor > The CBO can generate good execution plans only if the statistics are good as well, i.e. they must reflect existing data > Missing, not up-to-date, or wrong statistics can lead to poor execution plans > Object statistics are automatically gathered CBO: A Configuration Roadmap

11 Gathering object statistics (1) > Gather statistics for tables, columns and indexes at the same time» DBMS_STATS.GATHER_SCHEMA_STATS» DBMS_STATS.GATHER_TABLE_STATS > Use small estimate percentage (<10%)» Oracle increases the specified value if necessary» For very large tables reduce it to 1% or less > Activate table monitoring to refresh the statistics when data is changed (by default a threshold of 10% is used)» DBMS_STATS.ALTER_SCHEMA_TAB_MONITORING» STATISTICS_LEVEL = TYPICAL (default) > For partitioned tables gather statistics at all levels > For large tables gather statistics in parallel CBO: A Configuration Roadmap

12 Gathering object statistics (2) > Histograms are essential, gather them for all columns containing skewed data that are referenced in WHERE clauses» Even for non-indexed columns!» For simplicity use SIZE SKEWONLY, if it takes too much time try SIZE AUTO; if it is still too slow or the chosen number of buckets it not good, manually specify the list of columns» To get the most out of them, bind variables must not be used > The following command is a good starting point dbms_stats.gather_schema_stats( ownname => user, estimate_percent => 5, cascade => TRUE, method_opt => 'FOR ALL COLUMNS SIZE SKEWONLY', options => 'GATHER STALE' ); CBO: A Configuration Roadmap

13 Gathering object statistics (3) > All procedures that gather optimizer statistics no longer have hardcoded default values, instead, they are stored in the data dictionary» You can get and set them with the DBMS_STATS procedures GET_PARAM and SET_PARAM SQL> SELECT sname, nvl(to_char(sval1),spare4) value 2 FROM sys.optstat_hist_control$; SNAME VALUE CASCADE DBMS_STATS.AUTO_CASCADE ESTIMATE_PERCENT DBMS_STATS.AUTO_SAMPLE_SIZE DEGREE NULL METHOD_OPT FOR ALL COLUMNS SIZE AUTO NO_INVALIDATE DBMS_STATS.AUTO_INVALIDATE GRANULARITY AUTO > Check the definition of the scheduler window group MAINTENANCE_WINDOW_GROUP CBO: A Configuration Roadmap

14 System statistics (1) > System statistics are essential for a successful configuration > The CBO used to base its estimations on the number of I/O needed to execute a SQL statement (a.k.a. I/O cost model ) > With system statistics a new model, called CPU cost model, is enabled > They give information about the performance of the system where Oracle runs» Performance of the I/O subsystem» Average I/O size for multi-block read operations» Performance of the CPU > To use the I/O cost model, when system statistics have been gathered, specify the hint NO_CPU_COSTING CBO: A Configuration Roadmap

15 System statistics (2) > There are two kinds of system statistics > Without workload» Not really useful» They are automatically gathered at startup if they don t exist dbms_stats.gather_system_stats('noworkload'); > With workload» They should be gathered when the system is charged with a common workload» The instance is monitored for a period of time (interval) dbms_stats.gather_system_stats( gathering_mode => 'interval', interval => 30 ); Statistics are gathered over the next 30 minutes CBO: A Configuration Roadmap

16 System statistics (3) SQL> SELECT pname, pval1 2 FROM sys.aux_stats$ 3 WHERE 4 sname = 'SYSSTATS_MAIN'; PNAME PVAL CPUSPEEDNW IOSEEKTIM 10 IOTFRSPEED 4096 CPUSPEED 928 SREADTIM MREADTIM MBRC 9 MAXTHR SLAVETHR > Without workload» CPUSPEEDNW: CPU speed in millions of operations per second» IOSEEKTIM: I/O seek time [ms]» IOFRSPEED: I/O transfer speed [B/s] > With workload» CPUSPEED: CPU speed in millions of operations per second» SREADTIM: single-block read time [ms]» MREADTIM: multi-block read time [ms]» MBRC: average number of blocks read during a multi-block read operation» MAXTHR: max. throughput [MB/s]» SLAVETHR: max. slave throughput [MB/s] CBO: A Configuration Roadmap

17 OPTIMIZER_MODE > This is the most important parameter, unfortunately, too often it is not set, i.e. its default value is used» Default value is CHOOSE - ALL_ROWS is used if object statistics are available, otherwise RULE - In some situations ALL_ROWS has to be used even if no statistics are available (e.g. for partitioned tables)» Default value is ALL_ROWS > If fast delivery of the last row is important, ALL_ROWS should be used» Typical example: reporting systems and data warehouses > If fast delivery of the first row(s) is important, FIRST_ROWS_n should be used» Where n is [ ] row(s)» Typical example: OLTP systems CBO: A Configuration Roadmap

18 OPTIMIZER_FEATURES_ENABLE > Each database version introduces new features in the optimizer > OPTIMIZER_FEATURES_ENABLE can be used to set which version of the optimizer should be used» Valid values are database versions (e.g , 9.0.1, 9.2.0, )» A complete list of the features which are enabled/disabled by each version is available in the Oracle Database Reference manual» The default value is the current database version» Not all new features are enabled/disabled by this parameter, this means that even if you set it to in, for example, Oracle9i you won t get the optimizer! > It could be useful to set it when an application is upgraded to a new database version, otherwise leave it at the default value CBO: A Configuration Roadmap

19 DB_FILE_MULTIBLOCK_READ_COUNT (1) > DB_FILE_MULTIBLOCK_READ_COUNT specifies the maximum number of blocks that Oracle reads during a multi-block operation (e.g. full table scan) > Three common situations lead to multi-block reads that are smaller than the specified value» Oracle reads headers with single-block reads» Oracle never does an I/O that spans more extents» Oracle never reads a block that is already in the buffer cache > Multi-block reads are a performance feature, therefore, the parameter should be set to achieve the best performance» Higher values do not provide better performance in all cases!» It makes no sense to exceed the maximum physical I/O size CBO: A Configuration Roadmap

20 DB_FILE_MULTIBLOCK_READ_COUNT (2) > When system statistics with workload are not used this parameter has a direct impact on the full table scan costs, therefore, too high values lead to excessive full scans» Without system statistics I / O Cost FTS» System statistics without workload Blocks DFMBRC / Blocks I O Cost FTS 1+ A ln(dfmbrc) + B ln( DFMBRC) > When system statistics with workload are used, MBRC in SYS.AUX_STATS$ is used instead of DFMBRC to compute the cost of full table scans CBO: A Configuration Roadmap

21 DB_FILE_MULTIBLOCK_READ_COUNT (3) > A simple full table scan with different values gives useful information about the impact of this parameter and, therefore, assists in finding the best value Gain in % DFMBRC / MBRC I/O cost without I/O system cost statistics without I/O system I/O cost statistics without def. I/O cost def. system statistics I/O system cost statistics without System 1 def. system statistics I/O System System (max. cost 55 with 1 MB/s) def. system (max. 55 (max. 55 statistics MB/s) System 12 MB/s) System (max MB/s) System (max. 40 MB/s) (max. System 40 3 MB/s) (max. 58 MB/s) System 3 (max. System 584 MB/s) (max. 230 MB/s) CBO: A Configuration Roadmap

22 OPTIMIZER_DYNAMIC_SAMPLING (1) > The CBO used to base its estimations on statistics found in the data dictionary only > With dynamic sampling some statistics can be gathered during the parse phase as well, this means that some queries are executed against the referenced objects to gather additional information > The statistics gathered by dynamic sampling are not stored in the data dictionary, they are simply shared at cursor level > The value of the parameter OPTIMIZER_DYNAMIC_SAMPLING specifies how and when dynamic sampling is used» Range: from 0 to 10 (0=dynamic sampling disabled) CBO: A Configuration Roadmap

23 OPTIMIZER_DYNAMIC_SAMPLING (2) Level On which table? All non-analyzed tables, if at least one table» Is part of a join, subquery or non-mergeable view» Has no index» Has more blocks than the number of blocks used for the sampling All non-analyzed tables All tables which fulfill the level-2 criterion plus all tables for which a standard selectivity estimation is used All tables which fulfill level-3 criteria plus all tables which have single table predicates referencing more than two attributes # Blocks Param Hint All All CBO: A Configuration Roadmap

24 OPTIMIZER_DYNAMIC_SAMPLING (3) > Default values» If OPTIMIZER_FEATURES_ENABLE : 2» If OPTIMIZER_FEATURES_ENABLE = 9.2.0: 1» If OPTIMIZER_FEATURES_ENABLE 9.0.1: 0 > Level 1 and 2 are not very useful, in fact the tables should be analyzed on a regular basis!» An exception is when temporary tables are used, in fact, usually, no statistics are available for them > Level 3 and up are useful for improving selectivity estimations of predicates» If the CBO is not able to do correct estimations start by setting it to 4» It is possible to do such corrections with SQL profiles as well CBO: A Configuration Roadmap

25 PGA management > It is possible to choose between two methods to manage the PGA» Manual: the DBA has full control over the size of the PGA» Automatic: the DBA delegates the management of the PGA to Oracle > Except in the following situations it is recommended to use automatic PGA management» Very large PGA needed (>100MB, see Metalink note ) - Except if you want to set _PGA_MAX_SIZE» Fine tuning is needed» Shared server (former MTS) is used > Usually a larger PGA makes merge/hash joins and sort operations faster, therefore, you should devote the unused memory that is available on the system to it > The PGA size has an influence on the costs computed by the CBO CBO: A Configuration Roadmap

26 Manual PGA management > To enable manual PGA management the INIT.ORA parameter WORKAREA_SIZE_POLICY must be set to MANUAL > Then the following INIT.ORA parameters should be set» HASH_AREA_SIZE» SORT_AREA_SIZE» BITMAP_MERGE_AREA_SIZE > It is practically impossible to give advice about their value, anyway, here some general rules» Usually a minimal value of 512KB to 1MB should be used» For small PGAs, to take advantage of hash join, the HASH_AREA_SIZE should be at least 3-4 times the SORT_AREA_SIZE > Each parameter specifies the maximum amount of memory that can be used by each server process CBO: A Configuration Roadmap

27 Automatic PGA management > To enable auto PGA management the INIT.ORA parameter WORKAREA_SIZE_POLICY must be set to AUTO > Contrary to manual PGA management, the INIT.ORA parameter PGA_AGGREGATE_TARGET specifies the total amount of memory use for all PGAs, i.e. not for the single server processes > In some situations the PGA_AGGREGATE_TARGET could also be exceeded» It s a target, not a maximum value!» This usually happens when a value which is too small is specified CBO: A Configuration Roadmap

28 OPTIMIZER_INDEX_COST_ADJ > This parameter is used to change the cost of table accesses through index scans > Range of values (default 100)» Values greater than 100 make index scans more expensive» Values lower than 100 make index scans less expensive > Correction for an index range scan I / O Cost ( BLevel + ( LeafBlocks + ClustFactor) Selectivity) > Correction for an index unique scan I / O Cost ( BLevel + 1) OICA 100 OICA 100 > This parameter flattens costs and makes the clustering factor much less significant, therefore, small values should be carefully specified! With system statistics the default value is usually good CBO: A Configuration Roadmap

29 OPTIMIZER_INDEX_CACHING > This parameter is used to specify the expected amount (in percent) of index blocks cached in the buffer cache during a nested loop join» This parameter has no influence on index range scans! > Range of values (default 0)» Values greater than 0 make nested loops less expensive > Correction for an index range scan I / O Cost OIC + ClustFactor Selectivity 100 > Correction for index unique scan ( BLevel + LeafBlocks Selectivity) 1 OIC I / O Cost BLevel > With system statistics the default value is usually good CBO: A Configuration Roadmap

Core messages > The CBO works well if it is correctly configured! > Correctly configuring the CBO is not an easy job This is no good reason for not doing it!

30 Core messages > The CBO works well if it is correctly configured! > Correctly configuring the CBO is not an easy job This is no good reason for not doing it! > from Trivadis At the core it's all about data. > Understanding how each parameter changes the behavior of the CBO is essential CBO: A Configuration Roadmap

31 Thank you for your attention Questions?

Tuning with Statistics

Tuning with Statistics Wolfgang Breitling breitliw@centrexcc.com Agenda The DBMS_STATS Package Uses of DBMS_STATS beyond analyze The stattab Table Transferring Statistics with the stattab Table Means to