High Performance Oracle Optimizing Database I/O Dave Pearson Quest Software Copyright 2006 Quest Software
The Impact of Poor Performance Diagnostics and Optimization
The Impact of Poor Performance Diagnostics and Optimization
Quest Application Advantage Quest Application Advantage allows application teams to deliver, manage and control complex application environments Only Quest provides the deepest understanding of the application - from the end user to the database
Managing Production I/O Performance
Managing Production I/O Performance RESOLVE DISCOVER DIAGNOSE
Manage Discover, Diagnose and Resolve Poor Performance Database I/O
Database Sub-Systems Instance Transaction Configuration Storage
Poor I/O Performance Inefficient SQL generates excessive I/O Missing indexes spawn full table scans I/O hotspots cause CBC latch waits
Performance Analysis Indicates I/O Waits I/O waits are the primary contributor to overall performance degradation
Performance Analysis Indicates I/O Waits Sequential reads indicates index lookups
Database Sub-Systems Instance Transaction Configuration Storage
Poor I/O Performance Inefficient cache & pool sizes Inappropriate DB block size Sort area size set too low Slow redo log configuration Too few DBWR processes
Database Sub-Systems Instance Transaction Configuration Storage
Poor I/O Performance Wasted block space Index stagnation Row chaining Excessive extents Outdated statistics Default storage settings Index / Table Contention
Database Sub-Systems Instance Transaction Configuration Storage
Potential Causes of Poor I/O Performance Inefficient SQL Missing indexes I/O hot spots Inefficient cache & pool sizes Inappropriate DB block size Sort area size set too low Slow redo log configuration Too few DBWR processes Wasted block space Index stagnation Row chaining Excessive extents Outdated statistics Default storage settings Index / Table Contention Transaction Configuration Storage
Performance Resolution Workflow Instance Transaction Configuration Storage
Quest Central Performance Products Transaction Configuration Storage
Quest Central Performance Products Transaction Configuration Storage
Storage Optimization
Storage Optimization Analyze and Plan Capacity Predict Future Storage Needs Chart Trends in Capacity Usage Identify Storage Management Issues Overallocated Tablespaces Fastest Growing Datafiles Impending Out of Space Conditions Optimize Database Storage Identify Fragmentation within Tablespaces Reorganize Objects and Reclaim Space Optimize Storage Performance w/out Downtime Restructure for Improved I/O Balance
Storage Optimization Analyze and Plan Capacity Predict Future Storage Needs Chart Trends in Capacity Usage Identify Storage Management Issues Overallocated Tablespaces Fastest Growing Datafiles Impending Out of Space Conditions
Capacity Manager for Oracle Single view of storage across all databases, based on priority and/or criticality
Aggregate Allocation and Largest Databases
Database Allocation and Largest Tablespaces
Aggregate Growth and Fastest Growing Databases
Database Growth and Fastest Growing Tablespaces
Aggregate Allocation by Segment Type
What If Analysis Expected Changes
User Interface Print and save reports
In-Context Launch of LiveReorg
Storage Optimization Analyze and Plan Capacity Predict Future Storage Needs Chart Trends in Capacity Usage Identify Storage Management Issues Overallocated Tablespaces Fastest Growing Datafiles Impending Out of Space Conditions Optimize Database Storage Identify Fragmentation within Tablespaces Reorganize Objects and Reclaim Space Optimize Storage Performance w/out Downtime Restructure for Improved I/O Balance
Oracle Fragmentation Oracle databases experience 3 types of fragmentation Segment, Block and Row Segment Fragmentation Excessive extents place a load on the data dictionary Affects free space as well as allocated space Resolved with the adoption of LMT (locally managed tablespaces) in release 8i Note: LMT still not deployed in most SAP deployments
Block Fragmentation Block Fragmentation HWM (high water mark) vs Actual Used Wasted Space causes excessive I/O Excessive I/O slows query performance and application response time Before Reorg After Reorg
Affects of Block Fragmentation SAP Table: ANLV Table Size: 37.07MB Oracle: 9.2.0.2 Block Fragmentation: 59.25% Row Fragmentation: 0.0% Performance Metric Before Reorg After Reorg Percentage Difference Transactions Per Sec (TPS) 21.69 31.20 +43.85% Rows Returned During Test 6,678 11,165 +67.19% Average Time 0.819 0.539 +34.19%
Row Fragmentation Row Fragmentation Chained rows reside in multiple Oracle blocks The data in chained rows exceeds the block size, causing relocation of these rows to a new block Accessing chained rows requires traversal of each block in each chain A query to a chained row generates at least 100% more overhead than a query to an unchained row
Affects of Row Fragmentation Environment SAP Report: Tables Accessed: 10 Reorg Table: ZLSDRS0039 VBAP Oracle: 9.2.0.2 Table Size: 25.27GB Block Fragmentation: 0.00% Row Fragmentation: 3.96% Results Report generation time improved by an average of 27.90% (conducted over four runs)
How to LiveReorg a Database
How to LiveReorg a Database Reorg Table 1 Production Table 1. Create a copy of the table 2 2. Queue change transactions
How to LiveReorg a Database Reorg Table Production Table 1. Create a copy of the table 2. Queue change transactions and synch reorg table upon completion of table copy 2
How to LiveReorg a Database Production Table Original Table 1. Create a copy of the table 2. Queue change transactions and synch reorg table upon completion of table copy 3. Switch users to the new table by renaming the reorg table to the production name
Live Space Management Analysis and management of database storage Capacity planning and trending Exception reporting of space problems Before... Fast isolation of storage inefficiencies Live reorganisation and restructure
Live Space Management Analysis and management of database storage Capacity planning and trending Exception reporting of space problems... After! Fast isolation of storage inefficiencies Live reorganisation and restructure
Avoiding Downtime with LiveReorg SAP Table (DBERDZ) 1.86TB of data, 312GB of indexes Live Transactions Processed Inserts: 7,236,202 Updates: 5,276,000 Deletes: 90,492 Duration of Reorg (downtime saved) Table Copy: 95 hours Index Creation: 33 hours LiveReorg Continuous Performance Over 12.6M transactions No Downtime Eliminated 5.3 days of outage
Saving Storage with LiveReorg SAP Table (DBERCHZ) Aggressively archived database purge Storage Footprint Before Reorg: 2,986 GB (data and index) After Reorg: 2,084 GB (data and index) Storage Savings: 902 GB in a single reorg ( = 30%) Performance Total Reorg Time: 12.5 hours
Quest Space Manager with LiveReorg Performance and Storage Improved response times for online users Faster run times for large report jobs Optimised storage allocation Low Impact Light-weight collection of changed data Protects Availability Zero-downtime with online switch Validated by SAP proven scalability and support
Case Studies LiveReorg Customers Copyright 2006 Quest Software
I/O Performance... Airline: Resolving Row Fragmentation Environment SAP Report: Tables Accessed: 10 Reorg Table: ZLSDRS0039 VBAP Oracle: 9.2.0.2 Table Size: 25.27GB Block Fragmentation: 0.00% Row Fragmentation: 3.96% Results (after eliminating row chains) Report generation time improved by an average of 27.90% conducted over four runs
I/O Performance... Retail: Data Warehouse Elevated I/O Waits (single-block reads) over a sustained period Associated increase in Buffer Busy Waits Event Sec Waited Waits/Sec Waits single block read 55709.8 3013.88 27137223 buffer busy 9163.53 243.09 2191758 multi-block read 3540.46 26.7 240658 latch 1505.46 15.83 142658 direct path read 1487.25 15.01 135017 direct path write 343.58 10.64 95799
I/O Performance... Retail: Data Warehouse Buffer Busy Waits Undo header or block Configure more rollback segments Data block Verify transactions that are run against the affected object INSERTs Increase freelists on the affected object, reorg to apply to historical data (that preceded the decision to change value) SELECTs Optimise the storage of the affected object: minimise the number of extents and reorg the objects
I/O Performance... Telco: Billing Database Performance spike in Latch Free waits cache buffer chains latch free waits = hot blocks! Latch Sleeps Gets Misses Miss Rate Sleep Rate cache buffers chains 306104 5741823264 174484961 3.04 0.01 library cache 89544 130744559 208287 0.16 0.07 messages 11867 164919342 2463313 1.49 0.01 cache buffers lru chain 10597 51975384 330736 0.64 0.02 shared pool 10052 43690180 419392 0.96 0.02
Cache Buffer Chain Latch Waits Contention for CBC latches occurs when one or more of the following occur: Several users are frequently accessing the same data block Several users are frequently accessing several data blocks protected by the same CBC latch Frequently accessed cache buffers chains contain a large number of data block headers, slowing their scan time, and causing the CBC latch to be held for an excessive duration Potential Solutions Rewrite the application code to avoid hot blocks in the database Reorganise the objects with a higher PCTFREE to reduce the data density in each block Add more CBC latches
I/O Performance... Banking: ERP Application Elevated I/O Waits single-block reads + multi-block reads Increased Buffer Busy Waits
I/O Performance... University: E-Learning System Unbalanced I/O peaks indicate I/O imbalance indicator line set at 15 I/Os per second
I/O Performance... University: E-Learning System Unbalanced I/O resolved! I/O is balanced and peaks are eliminated after resolution
Quest Application Advantage Quest Application Advantage allows application teams to deliver, manage and control complex application environments Only Quest provides the deepest understanding of the application - from the end user to the database
Questions?
Thank You dave.pearson@quest.com Copyright 2006 Quest Software