Demystifying SQL Tuning: Tips and Techniques for SQL Experts

Demystifying SQL Tuning: Tips and Techniques for SQL Experts Mughees A. Minhas Director of Product Management, Database and Systems Management Sergey Koltakov Product Manager, Database Manageability

Outline Identifying Problem SQL Identifying Problem with the SQL Manual Automatic Preventing SQL Regressions

Top System SQL: ADDM Snapshots in Automatic Workload Repository Automatic Database Diagnostic Monitor (ADDM) Self-Diagnostic Automatic Diagnostic Engine inside Engine DB Provides database-wide performance diagnostic Self-diagnostic engine in the database High-load SQL IO / CPU issues RAC issues Identifies top SQL for the entire system in the last hour Shows SQL impact SQL Advisor System Resource Advice Network + DB config Advice Frequency of occurrence

Identify Top SQL using Top Activity Performance Page Identify Top SQL by Time CPU I/O Wait class Real-time analysis: ASH data Historical analysis: AWR data More granular analysis Top Activity Enables identification of transient problem SQL

Identify Problem SQL using AWR Compare Period Report Enables comparison of Top SQL from one period to another Makes analysis of SQL variations diagnosis easier AWR SQL capture configurable Important to establish baseline period for comparison AWR Baselines Out-of-box Moving Window Baseline

Outline Identifying Problem SQL Identifying Problem with the SQL Manual Automatic Preventing SQL Regressions

Causes of Poor SQL Performance Operational Issues Stale optimizer statistics CBO requires up-to-date statistics to optimize SQL execution Not easy to determine when stale statistics are impacting plan selection Poor plan selection due to incorrect optimizer estimates Manually hinting SQL a solution: Requires significant expertise Is time consuming and error-prone trial and error method Does not work for packaged applications

Causes of Poor SQL Performance Application Issues Missing access structures Designing correct partitioning strategy not easy Identifying useful indexes and MVs non-trivial Difficult to assess performance impact of new structures on DMLs Bad SQL design Sometimes the only real remedy is to restructure SQL Requires expertise, time, application knowledge

Identifying Problem with the SQL Manual Real-time SQL Monitoring ASH Report Automatic SQL Tuning Advisor SQL Access Advisor

Real-time SQL Monitoring Shows what s happening inside SQL execution Automatically monitors long running SQL > 5 secs of CPU or I/O All parallel statements Enabled out-of-the-box with no performance impact Monitors each SQL execution Exposes monitoring statistics Global execution level Plan operation level Parallel Execution level Guides tuning efforts

Demo Real-time SQL Monitoring

Identifying SQL Problem: Example 1

Identifying SQL Problem Big Plans

Identifying SQL Problem Example 2

Identifying SQL Problem Partially Parallelized

Identifying SQL Problem Example 3

Identifying SQL Problem PQ Skews

ASH Report ASH enables targeted performance analysis Facilitates transient problem diagnosis Helps answers questions like has this SQL gotten slower over time? ASH Report can be filtered by time, SID, SQL ID, Wait Class, Service, Module, Action, Client ID Performance Range * Time

ASH Report: SQL Main Sections

ASH Report: Top Events

ASH Report: Activity Over Time

Identifying Problem with the SQL Manual Real-time SQL Monitoring ASH Report Automatic SQL Tuning Advisor SQL Access Advisor

SQL Tuning Advisor Analysis SQL Tuning Advisor Comprehensive Analysis Recommendation SQL Statistics Analysis Gather Missing or Stale Statistics Access Structure Analysis (Single SQL) SQL Design Analysis Add Missing Indexes (B* Tree) Modify SQL Constructs Plan Tuning (SQL Profiling) Create a SQL Profile

SQL Profiles Contains auxiliary information for a SQL statement Customized optimizer settings: Based on past execution history (e.g., first_rows vs. all_rows) Compensates for missing or stale statistics Compensates for errors in optimizer estimates Doesn t require any change to the SQL text: Ideal for Packaged Apps SQL Profiling SQL Tuning Advisor submit Optimizer (Tuning Mode) create use SQL Profile After Database Users submit Optimizer (Normal Mode) output Well-Tuned Plan

Automatic SQL Tuning Nightly Packaged Apps Custom Apps SQL Tuning Advisor SQL Profiles Automatic implement Well-tuned SQL Auto Capture High-Load SQL SQL Analysis Report Manually implement Automatically captures highload SQL Automatically tunes SQL without changing application by creating SQL Profiles Automatically validates SQL Profiles by test executing them Automatically implements (optional) greatly improved SQL plans Automatically reports analysis Automatically runs during maintenance window

SQL Access Advisor Analysis SQL Access Advisor Comprehensive Analysis Recommendation SQL Add New Indexes (B* Tree, Bitmap) Access Structure Analysis (Workload) Add New MV s, Logs Add New Partitions (Hash, Range)

Resource Consumption SQL Tuning Advisor Limited mode: Resource consumption minimal Stats, index and SQL restructure analysis is cheap Average is less than 1 second per SQL statement Comprehensive mode: Resource consumption may be significant SQL Profiling can potentially consume non-trivial resources Roughly comparable to amount of resources/time consumed when executing SQL statement(s) SQL Access Advisor Resource consumption depends on size of SQL workload For small number of SQL, resource consumption not very high

Live vs. Remote Tuning Live system tuning Run SQL Tuning Advisor in Comprehensive mode Run SQL Tuning Advisor in Limited mode only if System does not have spare resources to tune SQL Run SQL Access Advisor for few SQL at a time Remote tuning Tuning remotely if Cumulative resources/time consumed by all SQL statements being tuned significant System cannot spare resources Use SQL Profile and SQL Tuning Set export/import capabilities

Remote SQL Tuning Performed to shield production system from performance impact of SQL advisors Production System Test System Move SQL Transportable STS Tune SQL Storage Move SQL Profile Transportable SQL Profile

Remote SQL Tuning Use Automatic SQL Capture feature of SQL Tuning Set (STS) to capture SQL Workload Always validate SQL Profiles before enabling them DBMS_SQLTUNE.ACCEPT_SQL_PROFILE (task_name => <tuning task name>, category => MY_CATEGORY ); ALTER SESSION SET SQLTUNE_CATEGORY= MY_CATEGORY ; For remote tuning, ensure test system is similar to production system Schema Data distribution Volume If test system smaller than production, set optimizer stats manually

Outline Identifying Problem SQL Identifying Problem with the SQL Manual Automatic Preventing SQL Regressions

Preventing SQL Regressions Gather statistics for all objects (dictionary and user objects) Volatile objects Gathers statistics when object at max size and then lock table Delete all statistics and lock table dynamic sampling will be used Restoring old optimizer statistics Used when new stats result in poor execution plan selection Scope: Table, schema and database History maintained for 30 days API: DBMS_STATS.RESTORE_TABLE_STATS Don't use the ANALYZE command Officially obsolete for optimizer statistics Cannot gather GLOBAL statistics for partitioned objects Cannot gather statistics for external tables, fixed tables, etc. Invalidates/recompiles all dependent cursors at once DBMS_STATS marks cursors as unusable and recompiles gradually

Preventing SQL Regressions Automatic statistics collection job Out-of-the box, runs in maintenance window Configuration can be changed (at table level) Gathers statistics on user and dictionary objects Parameters chosen automatically based on DML monitoring Column usage monitoring Iterative sampling Uses new collection algorithm with accuracy of compute and speed faster than sampling of 1% Incrementally maintains statistics for partitioned tables very efficient

Preventing SQL Regressions Extended Stats Extended Optimizer Statistics provides a mechanism to collect statistics on a group of related columns Full integration into existing statistics framework Automatically maintained with column statistics DBMS_STATS.CREATE_EXTENDED_STATS Pending Stats Allows validation of statistics before publishing Disabled by default To enable, set table/schema PUBLISH setting to FALSE DBMS_STATS.SET_TABLE_PREFS('SH','CUSTOMERS','PUBLISH','false') To use for validation ALTER SESSION SET optimizer_pending_statistics = TRUE; Monitor using view: USER_(TAB/COL/IND)_PENDING_STATS

Validating SQL Performance Validate all changes that impact SQL plans, e.g., statistics refresh Use SQL Performance Analyzer (SPA) to validate statistics refresh 1. Capture SQL workload in STS using automatic cursor cache capture capability 2. Execute SPA pre-change trial 3. Refresh statistics using PENDING option 4. Execute SPA post-change trial 5. Run SPA report comparing SQL execution statistics Before PUBLISHing stats Remediate individual SQL for plan few regressions Revert to old statistics if too many regressions observed SQL Performance Analyzer (SPA) 11.1 Storage Compare SQL Performance Analysis Report Send SQL to execute Send execution stats 10.2 Storage