1
<Insert Picture Here> Maximizing Database Performance: Performance Tuning with DB Time Kurt Engeleiter, John Beresniewicz, Cecilia Gervasio Oracle America
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 3
Agenda NEWS: DB Time is not news DB Time Fundamentals DB Time Method for Performance Tuning DB Time Based Tools ADDM Enterprise Manager 4
<Insert Picture Here> NEWS: DB Time is Not News 5
DB Time is Pervasive, Familiar, Evolutionary DB Time is the fundamental measure of Oracle performance Arrived at progressively but designed-in from 10g You are likely using DB Time methods already AWR and ASH reports Enterprise Manager DB Time is performance analysis science matured Wait event analysis, YAPP and tracing are all its ancestors 6
DB Time on the Database Home Page Host CPU Is host CPU-bound? Is this DB responsible? Active Sessions DB Time accumulation by CPU, User I/O, Wait 7
DB Time on the Performance Page Area under curve is equal to DB time Wait class breakdown indicates solution space 8
DB Time on the Top Activity Page Plots DB Time from ASH data Shows top DB Time consumers by SQL and Session DB Time 9
DB Time in AWR Report Time Model measures and accumulates DB Time through session instrumentation DB Time and DB CPU most important Also shows DB Time expended in various sub-operations 10
DB Time in AWR Report If you tune by Wait Events you are tuning by DB Time Top 5 Timed Events in the AWR report shows CPU time and non-idle waits for foregrounds 11
DB Time in the Grid Control DB Loadmap Maps DB Time across the enterprise Cell size proportional to DB Time 12
<Insert Picture Here> DB Time Fundamentals 13
Why Do We Care About Time? Human time is critical to the enterprise Systems performance affects business goals Human time + technology resource time Time is money Performance improvement means doing things faster Performance is always and only about time 14
DB Time Defined Total time in database calls by foreground sessions Includes CPU time, IO time and non-idle wait time DB Time is not called response time on purpose Common currency for Oracle performance analysis Database time is total time spent by user processes either actively working or actively waiting in a database call. 15
Session-level Concepts Database Time (DB Time) = Total time session spends in all database calls Active Session = Session currently spending time in a database call Average Activity of the Session (% Activity) = Ratio of time active to total wall clock time = Sum(Active Time) / Elapsed Time Browse books Read reviews for a book Add to cart Checkout TIME = time spent in database 16
Multiple Sessions DB Time = Sum of DB Time over all sessions Avg. Active Sessions = Sum of Avg. Activity over all sessions = Sum(DB Time) / Elapsed Time User 1 User 2 User 3 = time spent in database TIME User n 17
Visualizing DB Time User 1 User 2 User 3 User n t0 t1 4 3 2 1 Active Sessions over time TIME 18
Enterprise Manager Performance Page Active Sessions by wait class over time Colored area is the amount of DB Time DB Time on User I/O is always blue DB Time on CPU is always green 19
<Insert Picture Here> DB Time and System Performance 20
System Load and DB Time More users More database calls DB Time increases Larger transactions More SQLs per call DB Time increases DB time increases as system load increases. 21
System Performance and DB Time I/O performance degrades I/O wait time increases DB Time increases Application performance degrades Longer elapsed time per SQL DB Time increases DB time increases as performance degrades. 22
Host Performance and DB Time Host is CPU bound Foregrounds accumulate active run-queue time Wait event times are artificially inflated DB Time increases Tune for CPU before waits when CPU constrained 23
CPU Run-Queue and DB Time Recorded wait time Recorded wait time User 1 Db file sequential read Run-queue On CPU Db file sequential read Run-queue On CPU Actual wait time Actual wait time DB time is inflated when host is CPU-bound 24
<Insert Picture Here> The DB Time Method for Performance Tuning 25
The DB Time Method: Short Course Determine where or database time is just spent, ask and ADDM reduce it! 26
The DB Time Method: Process 1. Identify performance issue 2. Scope the issue 3. Set goals 4. Data capture AWR handles this 5. Investigate DB time distribution Identify largest potential improvement 6. Modify system to tune for largest gain 7. Evaluate against goals Performance tuning by removing excess DB time 27
Identify and Scope the Problem Big spike = big problem! Container (host) or database issue? 28
Investigate DB Time Distribution Identify excess DB time Look for large skews in time that may represent inefficiencies System scope: Resource limits is problem outside the DB? Resource contention (latches) Application scope: Service, module, action SQLID, rowsource Resource contention (enqueues) Session scope: Long-running SQL Resource contention (holding enqueues in uncommitted txns) 29
Investigate DB Time distribution Which wait classes are dominant? Top lists show skew in key dimensions (SQL, session, file, etc.) 30
Identify Time-saving Modifications System scope: Initialization parameter changes Add resources or capacity Application scope: SQL Tuning Advisor: SQL profiles Real-time SQL Monitoring Access Advisor: Physical schema layout Re-write SQL Session scope: Kill session for immediate contention relief Application design changes often long-term solution 31
Critical Success Factors Match solution scope to problem scope Don t tweak optimizer parameters before tuning SQL Address root causes, not symptoms Proceed iteratively one fix at a time Concurrent fixes should be orthogonal Measure and validate results at each successive step Stop when goals are met 32
Advantages of the DB Time Method Tunes exactly what affects users: Time wasted in database Data capture not necessary: Always on instrumentation framework (AWR, ASH) No requirement to reproduce problem (unlike tracing) Superior coverage of all scope levels Works for concurrency problems such as locking Tight integration with kernel-engineered tools Maturation and extension of well-proven methodologies 33
<Insert Picture Here> DB Time Based Tools: ADDM Enterprise Manager 34
Tools for Applying DB Time Method Two use-cases, one method: 1. Tuning steady-state performance Improve overall workload throughput or response time Best practice: use ADDM 2. Diagnosing transient performance problems Confirm and investigate reported performance issues Best practice: use EM real-time screens 35
Example: Using ADDM Users complain about bad performance AWR report indicates some unusual issues on system What to do? 36
What does the Internet advise? Let s search on enq: HW contention 9,000 hits and LOTS of discussion 37
Example: Following the links Many hits are other people asking the same question what do I do about this wait event? 38
Solution: ADDM in Enterprise Manager Performance Page shows a spike of unusual activity Click on the ADDM analysis icon 39
Solution: Using ADDM ADDM performance findings are DB Time based Ranked by impact = percent of DB Time affected Click on the issue with the highest impact 40
Solution: Using ADDM ADDM recommendations describe solutions for findings EM presents implement actions where possible 41
Best Practice: Enterprise Manager Real-time Interface Transient (sub-hour) or immediate time scope Requires interactivity of UI Click on the big stuff Data visualizations display skew directly Takes some expertise to separate symptoms from root causes 42
Click on the Big Stuff Follow the DB Time Spike in DB Time signals possible performance issue Click through to Top Activity for more details 43
Follow the DB Time Top Activity Drilldown Top Activity Page confirms problem and shows highly skewed DB Time by SQL ID. Click on top SQL ID for further details. 44
Click on the Big Stuff Follow the DB Time SQL Details confirms this SQL produced the spike in DB Time and shows parallel execution. Click on SQL Monitoring to see DB Time by execution plan steps. 45
Follow the DB Time SQL Execution Details 46
DB Time Method Summary DB time is the fundamental performance metric The method allows DB time analysis at many scopes Proper scoping of problems and solutions is critical to success DB time based diagnosis removes value judgments Scientific method, not sorcerer s magic Performance improvement means doing the same work in less DB Time 47
Oracle Enterprise Manager 11g Resource Center Access Videos, Webcasts, White Papers, and More Oracle.com/enterprisemanager11g 49
Additional Oracle Enterprise Manager Sessions Wednesday, Sept. 22 10:00 a.m. - Manage the Manager: Diagnosing and Tuning Oracle Enterprise Manager 11:30 a.m. - Maximizing Database Performance: Performance Tuning with DB Time 11:30 a.m. - Make Upgrades Uneventful Using Oracle Enterprise Manager and My Oracle Support 12:30pm Extracting Real Value from Your Data with Apache Hadoop 1:00 p.m.- Reducing the Risk of SOA Transactions 1:00 p.m. - SQL Tuning Roundtable with Oracle Gurus 4:45 p.m.- Strategies for Monitoring Large Datacenters with Oracle Enterprise Manager 4:45 p.m.- Oracle SOA Management Best Practices, Tips, and Techniques 4:45 p.m.- Oracle E-Business Suite Technology: Vision, Release Overview, Product Roadmap Location Moscone S Rm 102 Moscone S Rm 104 Moscone S Rm 310 Hilton Hotel, Plaza B Marriott Marquis, Salon 6 Moscone S Rm102 Moscone S Rm102 Moscone W L3, Rm 3018 Moscone W L3, Rm 3002 / 3004 50
Additional Oracle Enterprise Manager Sessions Thursday, Sept. 23 9:00 a.m. - Oracle WebLogic Server Management for Oracle DBAs 9:00 a.m. - Enabling Database as a Service Through Agile Self- Service Provisioning 9:00 a.m. - Reduce TCO with Oracle Application Management Pack for Oracle E-Business Suite 10:30 a.m. - Best Practices for Managing Your PeopleSoft Applications 10:30 a.m. - Oracle Enterprise Manager Grid Control Deployment Best Practices 10:30 a.m. - Managing Sun SPARC Servers with Oracle Enterprise Manager Ops Center 10:30 a.m. - Heterogeneous Data Masking: Oracle, SQL Server, and DB2 Database Best Practices 12:00 p.m. - Scalable Enterprise Data Processing for the Cloud with Oracle Grid Engine 12:00 p.m. - Spot Problems Before Your Users Call: User Experience Monitoring for Oracle Apps 12:00 p.m. - Reduce Problem Resolution Time with Oracle Database 11g Diagnostic Framework Location Marriott Marquis, Salon 9 Moscone S. Room 102 Moscone W L2, Rm 2024 Marriott Hotel, Golden Gate A Moscone S. Room 102 Moscone S. Room 252 Moscone S. Room 306 Moscone S. Room 310 Marriott Hotel, Golden Gate A Moscone S. Room 102 51
Additional Oracle Enterprise Manager Sessions Thursday, Sept. 23 1:30 p.m. - Patching Enterprisewide Databases: Automation Techniques and Real-World Insights 1:30 p.m. - Managing User Experience: Lessons from ebay 1:30 p.m. - Deep Java Diagnostics and Performance Tuning: Expert Tips and Techniques 1:30 p.m. - Oracle Enterprise Manager Configuration Management Unleashed: Top 10 Expert Tips 1:30 p.m. - Oracle Enterprise Manager Security Best Practices 3:00 p.m - The X-Files: Managing the Oracle Exadata and Highly Available Oracle Databases 3:00 p.m. - Monitoring and Diagnosing Oracle RAC Performance with Oracle Enterprise Manager Location Moscone S. Room 310 Marriott Hotel, Golden Gate A Marriott Marquis, Salon 9 Marriott Marquis, Salon 6 Moscone S. Room 102 Moscone S. Room 102 Moscone S. Room 310 52
Oracle Enterprise Manager Hands On Labs Monday September 20, 2010 3:30 p.m. - 4:30 p.m. Database Performance Diagnostics and Tuning Marriott Hotel, Salon 12/13, YB Level 5:00 p.m. - 6:00 p.m. Provisioning, Patch Automation, and Configuration Management Pack Marriott Hotel, Salon 12/13, YB Level 5:00 p.m. - 6:00 p.m. Oracle Application Mgmt. Pack for Oracle E-Business Suite: Monitor/Clone Marriott Marquis, Nob Hill Tuesday September 21, 2010 11:00 a.m.-12:00 p.m. Using Oracle Application Change Management Pack for Oracle E-Business Suite Marriott Marquis, Nob Hill 12:30 p.m.-1:30 p.m. Database and Application Testing Marriott Hotel, Salon 12/13, YB Level 2:00 p.m. - 3:00 p.m. Oracle Fusion Middleware Management Marriott Hotel, Salon 12/13, YB Level 3:30 p.m. - -4:30 p.m. Provisioning, Patch Automation, and Configuration Management Pack Marriott Hotel, Salon 12/13, YB Level Wednesday September 22, 2010 4:45 p.m. - 5:45 p.m. Database and Application Testing Marriott Hotel, Salon 12/13, YB Level 4:45 p.m. - 5:45 p.m. Oracle Application Mgmt. Pack for Oracle E-Business Suite: Monitor/Clone Marriott Marquis, Nob Hill Thursday September 23, 2010 9:00 a.m. - 10:00 a.m. Database Performance Diagnostics and Tuning Marriott Hotel, Salon 12/13, YB Level 10:30 a.m. - 11:30 a.m. Oracle Fusion Middleware Management Marriott Hotel, Salon 12/13, YB Level 53
Oracle Enterprise Manager Demogrounds D E M O T I T L E Oracle Real Application Testing: Database Replay Oracle Real Application Testing: SQL Performance Analyzer Self-Managing Database: Automatic Performance Diagnostics Self-Managing Database: Automatic Fault Diagnostics Self-Managing Database: Automatic Application and SQL Tuning Application Quality Management: Application Testing Suite Real User Monitoring with Oracle Enterprise Manager Siebel CRM Application Management Real User Monitoring with Oracle Enterprise Manager Oracle WebLogic Server Management and Java Diagnostics SOA Management with Oracle Enterprise Manager Oracle Business Transaction Management Push Button Provisioning and Patch Automation Smart Configuration Management Oracle Enterprise Manager Ops Center Managing the Enterprise Private Cloud System Management, My Oracle Support, and Oracle Enterprise Manager Self Managing Database: Change Management for DBAs Oracle Enterprise Manager: Complete Datacenter Management Self-Managing Database: Data Masking for DBAs L O C A T I O N Moscone West Moscone West Moscone West Moscone West Moscone West Moscone South - S022 Moscone South - S021 Moscone South - S024 Moscone West Moscone West Moscone West Moscone West Moscone West Moscone West Moscone West Moscone West Moscone West Moscone West Moscone West Moscone West 54
55