IOUG Oracle EM SIG TechCast Performance Tuning your DB Cloud in OEM 12c Cloud Control

Transcription

1 IOUG Oracle EM SIG TechCast Performance Tuning your DB Cloud in OEM 12c Cloud Control 360 Degrees Tariq Farooq Michael R. Ault A BrainSurface Presentation

2 Tariq Farooq Oracle ACE Director Oracle Technologist for 20+ years: Architect, Author, Speaker, Blogger, Forumizer & Community Organizer Focus Areas: IT Architecture, Core DBA/DMA, RAC, Exadata, EBS, Fusion Apps, Virtualization, Cloud Computing Co-Author: Expert Oracle RAC 12c Currently authoring multiple books on Exadata, E-Business Suite and Building Oracle Database Clouds Oracle Certified Expert Real Application Clusters Oracle Certified Professional DBA 8i, 9i, 10g, 11g & 10g Apps DBA Oracle Certified Professional Internet Application Developer 2, 6i, 9i Oracle Certified Professional E-Business Suite 11i Founder of BrainSurface a Collaborative Social Networking, IT Knowledge ConnectSpace for the Oracle, Java & MySQL domains; catering FREE services to thousands of signed-up users President & Conference Chair VirtaThon The largest virtual conference for the Oracle, Java & MySQL domains BootCamp, Track Manager & Conference Committee Member IOUG COLLABORATE12/13 Virtualization and Cloud Computing Tracks Founding President - IOUG Cloud Computing Special Interest Group (SIG)

3 About Michael R. Ault Oracle Guru - Nuclear Navy 6 years - Nuclear Chemist/Programmer 10 years - Kennedy Western University Graduate - Bachelors Degree Computer Science - Certified in all Oracle Versions Since 6 - Oracle DBA, author, since Author of 25+ Oracle books

4 Disclaimer This views/content in this document are those of the author(s) and do not necessarily reflect that of Oracle Corporation and/or its affiliates/ subsidiaries. The material in this document is for informational purposes only and is published with no guarantee or warranty, express or implied.

5 IOUG Cloud Computing Special Interest Group (SIG) Sign-up today at & on Dedicated to communication and information around all areas of Oraclecentric Cloud Computing Learn from thought leaders in the Oracle Cloud Computing community Openly discuss and share the key challenges, options and best practices around key areas of Cloud Computing from the perspective of Oracle Access year-round education through webinars, online symposia, COLLOBORATE and Oracle OpenWorld Get the user and vendor perspective all in one place!

6 Session Abstract In this session, the presenters/authors talks about their experience, lessons learned, tips/tricks, recommendations, best practices, and gotchas on how to effectively use Oracle Enterprise Manager Cloud Control 12c for easy, intuitive, and quick-to-learn database performance tuning. A limited deep-dive of AWR is also presented.

7 SQL/Command-line generation functionality for some tasks. Purpose & Objectives: Benefits & Advantages How to effectively leverage OEM Cloud Control Oracle Database Performance Tuning? Easy-to-do/implement. Low learning curve. Efficient & Effective (Out-of-the-box In-Built Best Practices). Complex DBA administration/ management/ tuning with point-and-click functionality (GUI).

8 OEM Cloud Control: The GUI vs. Command-line debate GUI vs. Command-line: The classical debate spanning decades of back-andforth arguments/logic/reasoning. OEM Cloud Control is an advanced & sophisticated Management/ Monitoring/ Tuning system (tool) and over time (somewhat already has) will result in a paradigm shift in the ways that, DBAs work (perform their daily duties).

9 OEM Cloud Control: The GUI vs. Command-line debate Presenter/Author wrote an article in 2008, stating that, Oracle Enterprise Manager (OEM) would be instrumental/key factor in the next generations of DBAs being heavily Tool/GUI-dependent (predicted to be 5-15 years from then). This phenomenon is now happening. e.g. Advancement in voice recognition will ultimate result in Voice as the medium of instruction rather than the keyboard. This requires a certain degree of sophistication in the management/ administration software; something that OEM Cloud Control pushes for and, provides as a base platform already.

10 Oracle DB Tuning with OEM 12c Cloud Control: Synopsis & Overview Oracle Enterprise Manager Cloud Control is Oracle s end-to-end management & monitoring tool of the enterprise IT infrastructure. Re-engineered thin-client web-based architecture with 10g & up. Base OEM Cloud Control is FREE. Requires Diagnostics & Tuning Pack for enhanced & effective monitoring, troubleshooting & tuning of your Oracle database. OEM 12c (Cloud Control) announced/released at OOW11: New look & feel, based on Weblogic Server. Tonnes of new features for DB Tuning/ Monitoring/ Management. Plugin based architecture. Geared towards the new & emerging Cloud Computing paradigm/ space.

11 Oracle DB Tuning with OEM 12c Cloud Control: Synopsis & Overview 360 Degree visual overview/dashboard of entire database & overall IT ecosystem. OEM Cloud Control facilitates & accelerates identification & fixing of DB bottlenecks, problematic SQL statements, Locks, Contention, Concurrency & other potential crisis situations including monitoring/logging-into HUNG databases. Saves DB Administration/ Management/ Tuning TIME: thereby, reducing DOWN/SLOW time for the business.

12 OEM DB Control vs. Cloud Control: Understand What is What, and Why do You need What? DB Control installed by default for each database (Management of single database ONLY) and, is a flavor of OEM very similar to Cloud Control. OEM Cloud Control used to manage the entire IT ecosystem including Oracle & Non-Oracle targets. e.g. Cloud Control features/functionality much more enhanced & vast as compared to DB Control e.g. Setting up Data Guard within OEM Cloud Control using the GUI wizard-like screens is not available in OEM DB Control.

13 Oracle DB Tuning in OEM 12 Cloud Control: The tool Top-Down & Bottom-Up Tuning Tuning your Oracle DB is not ONLY about tuning SQL (Although it is significantly important)! Identify system-level (IO, CPU, Memory etc.) issues/ abnormalities/ crisis situations. Work your way from top-to-bottom by leveraging Advisors & Advisories built into the Oracle Database Server from an intuitive, feature-rich & easy-to-learn GUI within OEM Cloud Control.

14 Oracle DB (Performance) Tuning in OEM 12c: Visual Monitoring, Analysis & Tuning It s all about Graphs, Charts & Colors (OR Pictures)! The DB Performance Menu/Page is the heart and soul of the Visual Monitoring/Analysis process/methodology. Tips/Tricks/Methodologies: Familiarize yourself with the Graphs and, learn to understand them in detail over time. Get accustomed to your DB s baselines during peak-time activity time-frames. Know what the colors represent and, the overall global picture that, they want to communicate to you as a DBA.

15 Oracle DB (Performance) Tuning in OEM Cloud Control: Building Reflexes DB Performance page: One glance should translate into the overall health of the system in our (human) brain; In other words, comprehension of this screen over time, should translate into almost-reflex action (and follow-up steps if necessary) as a DBA. A good analogy in explaining this Paradigm/ Acquired Behavior is our driving skills/experiences and, how we prepare for/react to our car s dashboard and, it s various dials/gauges in (often) split-second response times.

16 Oracle DB (Performance) Tuning in OEM Cloud Control: DB Home page OEM 12c > Targets > Select DB: Home Page gives Performance, Resources & SQL Summary

17 Oracle DB (Performance) Tuning in 12c: Performance Menu : The Star-Gate/Launching Point Performance Menu > Performance Home page: The star-gate of Oracle DB Tuning in OEM Cloud Control: A very structured & useful organization of Performance-centric links/resources. Significant improvement over 10g/11g OEM Grid Control. Will ask you to login with SYSDBA or DBSNMP monitoring credentials.

18 Oracle DB (Performance) Tuning in OEM 12c: Real-Time & Historical Perspective Performance Home Page Filter by: Real-Time Refresh/Historical: Shows a historical perspective of your DB spread over periodic time intervals.

19 Oracle DB (Performance) Tuning in OEM 12c: Understanding the Colors Performance Menu Understanding the Colors and the information/severity level they represent: GREENish / BLUEish colors. GRAYish colors. BROWNish colors. REDish colors.

20 Oracle DB (Performance) Tuning in OEM 12c: The three A s The 3 A s of DB Performance Tuning in OEM Cloud Control: Active Session History (ASH). Automatic Workload Repository (AWR) The Black Box of the Oracle Database Server family. Automatic Database Diagnostic Monitor (ADDM).

21 Oracle DB (Performance) Tuning in OEM 12c: AWR/ADDM Historical ADDM/AWR drill-down: Click on any of the Green Tick-marks to access AWR/ADMM.

22 Oracle DB (Performance) Tuning in OEM Cloud Control: AWR/ADDM Running ADDM implicitly/correspondingly grabs a new AWR Snapshot: Performance Menu: Click on any of the Green Tick-marks to access the historical AWR/ADMM: How cool & easy is that??? OR Click on Run ADDM Now to get a fresh one.

23 Oracle DB (Performance) Tuning in OEM 12c: AWR/ADDM ADDM Report shown both in Drill-down format as well as plain text.

24 Oracle DB (Performance) Tuning in OEM 12c: AWR/ADDM On ADDM Report click on View Snapshots to show AWR snapshots/reports.

25 Oracle DB (Performance) Tuning in OEM 12c: AWR/ADDM Entire color-coded AWR report available for viewing/downloading/analysis.

26 AWR - What is this beast??? A background process Takes snapshot of high-cost SQL every hour By default, takes snapshots of statistics every hour A set of tables A set of reports

27 AWR - Preparation for Analysis Know your systems normal performance fingerprint Be familiar with Concepts and Tuning Guides Have normal AWR/Statspacks for comparison

28 AWR - Top-Down Approach Report starts with settings overview Next provides Top-5 waits Use the Waits to guide further investigation

29 AWR - Report Header WORKLOAD REPOSITORY report for DB Name DB Id Instance Inst Num Startup Time Release RAC AULTDB aultdb Aug-08 10: YES Host Name Platform CPUs Cores Sockets Memory(GB) aultlinux3 Linux IA (32-bit) Snap Id Snap Time Sessions Curs/Sess Begin Snap: Aug-08 12:00: End Snap: Aug-08 13:00: Elapsed: (mins) DB Time: (mins) Cache Sizes Begin End ~~~~~~~~~~~ Buffer Cache: 1,312M 1,312M Std Block Size: 8K Shared Pool Size: 224M 224M Log Buffer: 10,604K

30 AWR - Know Your Load Type! Online Transaction Processing Few reads Many writes Many small transactions Look for redo/undo and sequential read issues Decision Support/Data Warehouse Many reads Few writes (other then possible temp) Few transactions Look for sort/workarea and scattered read issues Mixed or Hybrid

31 AWR - Load Profile Section Load Profile Per Second Per Transaction Per Exec Per Call ~~~~~~~~~~~~ DB Time(s): DB CPU(s): Redo size: ,461.8 Logical reads: 6, ,396.7 Block changes: Physical reads: 2, ,317.8 Physical writes: User calls: Parses: Hard parses: W/A MB processed: 932, ,868,990.9 Logons: Executes: Rollbacks: Transactions: 0.3

32 AWR - What Are Your Efficiencies Should be close to 100% Parse issues usually are a result of: Bad bind variable usage Insufficient memory Will also be co-indicated by low percentage of memory for multiple SQL execution

33 AWR - Load Profile Section Instance Efficiency Percentages (Target 100%) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Buffer Nowait %: Redo NoWait %: Buffer Hit %: In-memory Sort %: Library Hit %: Soft Parse %: Execute to Parse %: Latch Hit %: Parse CPU to Parse Elapsd %: 0.00 % Non-Parse CPU: Shared Pool Statistics Begin End Memory Usage %: % SQL with executions>1: % Memory for SQL w/exec>1:

34 AWR - Top 5 Waits Section Critical to look closely at this section Use highest wait times to guide investigation DB FILE type waits physical IO BUFFER type waits Logical IO LOG type waits Redo related PX Parallel Query GC Global Cache (RAC related) Undo Undo or rollback segment related

35 AWR - Top 5 Waits Section Top 5 Timed Foreground Events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Avg wait % DB Event Waits Time(s) (ms) time Wait Class db file sequential read 465,020 3, User I/O DB CPU db file parallel read 2, User I/O db file scattered read 15, User I/O gc current block 2-way 108, Cluster

36 AWR - DB File Type Waits DB File Sequential Reads memory starvation, non-selective indexes DB File Scattered Reads full table scans, insufficient indexing Direct Path Writes Appends, data loads Direct Path Reads Parallel slaves used to retrieve data DB File Parallel Writes Backup and partition use DB File Parallel Reads Partition use DB File Single Write File header writes, excessive data files Direct path read temp Temp file activity (sorts, hashes, temp tables, bitmaps) Direct path write temp Temp file activity (sorts, hashes, temp tables, bitmaps)

37 AWR - Buffer Type Waits latch: cache buffers chains Hot blocks, check for hot objects free buffer waits Insufficient buffers, processes holding buffers too long, IO subsystem over loaded buffer busy waits See what is causing them further along in report gc buffer busy Overloaded interconnect, find problem objects and tune log buffer space High load, too small a log buffer, increase log buffer size latch: cache buffers lru chain Freelist issues, hot blocks latch: cache buffer handles Freelist issues, hot blocks buffer busy - See what is causing them further along in report no free buffers Insufficient buffers, dbwr contention

38 AWR - Log Type Waits log file parallel write Look for log file contention log buffer space Look at increasing log buffer size log file switch (checkpoint incomplete) May indicate excessive db files or slow IO subsystem log file switch (archiving needed) Indicates archive files are written too slowly log file switch completion May need more log files per thread log file sync Could indicate excessive commits

39 AWR - PX Type Waits PX Deq: Msg Fragment PEM maybe too small PX qref latch - Data is produced faster than it is consumed, look at PEM PX Deq Credit: send blkd Look at PEM size and parallel query into non-parallel DML

40 AWR - GC Events gc cr multi block request Full table or index scans gc current multi block request Full table or index scans gc cr block 2-way Blocks are busy in another instance, check for block level contention or hot blocks gc cr block 3-way Blocks are busy in another instance, check for block level contention or hot blocks gc cr block busy Blocks are busy in another instance, check for block level contention or hot blocks gc cr block congested cr block congestion, check for hot blocks or busy interconnect gc cr block lost Indicates interconnect issues and contention gc current block 2-way Blocks are busy in another instance, check for block level contention or hot blocks gc current block 3-way Blocks are busy in another instance, check for block level contention or hot blocks gc current block busy Block is already involved in GC operation, shows hot blocks or congestion gc current block congested current block congestion, check for hot blocks or busy interconnect gc current block lost - Indicates interconnect issues and contention

41 AWR - Undo Events undo segment extension If excessive, tune undo latch: In memory undo latch If excessive could be bug, check for your version, may have to turn off in memory undo wait for a undo record Usually only during recovery of large transactions, look at turning off parallel undo recovery.

42 AWR - What Next? Determine wait events of concern Drill down to specific sections of report for deeper analysis Use custom scripts, ADDM and Ash to investigate issues

43 Oracle DB (Performance) Tuning in OEM 12c: ASH Performance Page > Running on-the-fly ASH Reports.

44 Oracle DB (Performance) Tuning in OEM 12c: ASH Color-coded ASH Report

45 Oracle DB (Performance) Tuning in OEM 12c: Advisor Central Performance Menu> Advisor Home

46 Oracle DB (Performance) Tuning in OEM 12c: Advisor Central: SQL Advisors Performance Menu> SQL: SQL Tuning Advisor SQL Performance Advisor SQL Access Advisor SQL Tuning Sets SQL Plan Control Optimizer Statistics

47 Oracle DB (Performance) Tuning in OEM 12c: Advisor Central: SQL Advisors Advisor Central: SQL Advisors

48 Oracle DB (Performance) Tuning in OEM 12c: Advisor Central: SQL Advisors Quick & Easy: Potentially rapid/game-changer gains with a point-and-click functionality.

49 Oracle DB (Performance) Tuning in OEM 12c: Advisor Central: Memory Advisors Performance Menu> Memory Advisors

50 Monitoring, Tuning & Troubleshooting: Top Activity Performance Menu > Top Activity Shows Top SQL & Top Sessions. Rapidly/Visually shows the Top SQL & Top Sessions with drilldown functionality.

51 Monitoring, Tuning & Troubleshooting: Real Time ADDM Performance Menu > Real Time ADDM Connection Modes: Normal Mode Diagnostic Mode Analyze and monitor current DB performance/activity Resolve problems like Deadlocks, Hangs etc. Can login to a HUNG database

52 Monitoring, Tuning & Troubleshooting: Other Important Links/Features Performance Menu> ASH Analytics AWR > Compare Period ADDM SQL > SQL Plan Control Blocking Sessions

53 Monitoring, Tuning & Troubleshooting: SQL Monitoring Performance Menu > SQL Monitoring

54 RAC Cluster Databases: Monitoring, Tuning & Troubleshooting RAC Cluster Databases: Performance Menu: Presenting a global holistic picture of the entire DB Cluster as a System/Service.

55 RAC Cluster Databases: Monitoring, Tuning & Troubleshooting Cluster Databases: Cluster Cache Coherency : The main link/entry page for RAC DB Monitoring/Tuning.

56 Additional Monitoring, Tuning & Troubleshooting: Emergency Monitoring Performance Menu > Emergency Monitoring

57 Monitoring & Diagnostics: Useful Links Oracle Database menu > Monitoring Diagnostics

58 Summary & Q/A Oracle Enterprise Manager 12c Cloud Control is a Solid, Powerful, Robust & Enterprise-Grade built-for- Oracle-by-Oracle tuning Go-To Solution that, provides intuitive out-of-the-box administration, management & performance tuning of Oracle databases in addition to end-to-end management of the entire enterprise IT ecosystem Sign-up at: Contact: tariqdotfarooqatbrainsurfacedotcom