IBM Optim Performance Manager for DB2 on LUW V4.1

Transcription

1 IBM Optim Performance Manager for DB2 on LUW V4.1 Holger Karn Senior Technical Staff Member, Data Studio development

2 Disclaimer Copyright IBM Corporation All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. IBM, the IBM logo, ibm.com, and DB2 are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at Other company, product, or service names may be trademarks or service marks of others.

3 New Optim Performance Manager for DB2 on LUW (OPM) Successor product of DB2 Performance Expert New easy-to-use browser interface To access health and availability information outside of the office Overall health summary with immediate problem detection and drill-down analysis Reduce up and running costs with single simplified install per site Integration with IBM Optim Query Tuner Get immediate expert advice and resolve problematic SQL statements Integration with Optim Development Studio Detect line of code containing SQL statement for immediate problem resolution Immediate resolution of common performance problems with OPMs problem focused dashboards Integrated workload management UI Let s you assign resources to the right tasks Enhanced end-to-end monitoring Support for more than just JCC clients with new CLI support Out of the box support for Cognos, SAP, SQW, and DataStage Insights into what workload is doing inside database (I/O, locking, etc.) New graphical report engine

4 Optim Performance Manager Benefits Reduce Costs Quicker time to resolution 3 step workflows rather than many screens Close loop process for typical performance problems Integration shortens time to proactively manage performance Same UI OPM lets you check your WLM configuration and helps in creating initial definitions Problematic SQL statements can easily be identified and will then be analyzed in a one-click step with Query Tuner purequery tells you exactly where problematic SQL statements are coded Enable transition from heritage Performance Expert to OPM Packages both heritage PE interface and new OPM UI Prevent outages and reduce application slowdown Reduce Risk Enhanced WLM management through? Dashboard Alerting capability Historical performance data available to enable capacity planning Visualizing and aggregation for key performances Averages over time Set granularity so can scale on time and focus Decrease time to data value Increasing current staff expertise through IBM knowledge Close loop process for typical performance problem analysis QA Testing is easier so increase confidence to go into production Historical performance data available to enable capacity planning

5 Easy configuration using configuration templates OPM Configuration 1 Specify DB location... is mentioned here because DB2 Performance Expert configuration was difficult 2 Select configuration template New templates allow one-click configuration: - User can choose from various templates for typical installations - Each templates contains then suitable configuration - Only minor adaption would have to be done (if any) 3 Press Finish

6 Getting a health overview the Health Summary OPM Health Summary... gives you a quick overview about the health and utilization of ALL your databases. alerts for performance areas will tell you immediately if something is critical and needs further attention alerts can be send automatically via to database staff, even if not being logged on alert sensitivity can be adjusted a according to system and workload configuration can easily be cloned to other databases chronological list of alerts also available

7 Getting a health overview the Overview dashboard OPM Overview dashboard Icons show alert status of a KPI... gives you a quick overview about the health and utilization of a database. Key-performance-indicators (KPIs) will tell you what s going on alerts for critical areas will tell you immediately if something is critical and needs further attention Vertical lines indicate alert sensitivity detailed dashboards for each of the areas will let you then isolate and analyze a problem historical information lets you go back in time and see how a problem arised, or if problems occurred in the past

8 Getting a health overview the overlays Buttons let you get more information Charts tell you what the average was and if there was an outlier partition and which partition it was

9 OPMs inflight dashboards OPM in-flight dashboards... will let you quickly isolate and analyze typical database performance problems Memory Check memory consumption of system and see which areas consume how much memory and if they really need it. Buffer Pool and I/O Check and tune database I/O Active SQL Isolate missbehaving currently running queries, stop them, or tune them with Optim Query Tuner Logging Check and tune log performance Locking Identify, analyze, and fix deadlocks, timeouts, and lock waits Utilities Plan execution of utilities and identify failures System Check system resources Workload Get overviw about database utilization

10 Locking dashboard OPM locking dashboard... let s you see the real application which is causing trouble or having problems. auto-detects typical applications, such as WebSphere apps, DataStage jobs, Cognos reports, SQL Warehouse operations, or SAP transactions helps to differentiate easily between critical problems and just background noise uses application terms instead of database terms to point to root cause area (such as report name, or DataStage step ID inside job) shows you also the statement history which lead to a deadlock problem

11 Buffer Pool and I/O dashboard OPM BP & I/O dashboard... lets you check buffer pools, table spaces, and tables helps identifying hot objects and moving them to dedicated buffer pools lets you check the approprite size of a buffer pool lets you check the disk space and container definition of table spaces Double clicking or using the Show Contained Objects button lets you drill down into contained objects Buffer Pool Table Space Table

12 Memory dashboard OPM Memory dashboard... lets you check which area inside DB2 is using how much memory shows memory usage by instance, database, application shared usage, and application private usage shows you how those areas are configured for a partitioned DB you also see if you have outlying partitions depending on the area you also get health information (hit ratios) and utilization information

13 Active SQL dashboard OPM Active SQL dashboard... lets you easily check if rogue queries are eating up your database resources queries can be stopped or whole connection be canceled if necessary statement execution plan can be analyzed via Optim Query Tuner for a missing index, MQT, or outdated statistics historical data allows you to better plan execution of heavy statements

14 Tune queries with Optim Query Tuner Advisors give you hints how to tune the query Statistics advisor Provides advice for missing, outdated and conflicting statistics, and suggests additional statistics that you can gather to improve query performance. RUNSTATS jobs are provided with the recommendations. Index advisor Provides advice on new indexes that may improve performance as well as the ability to define and validate user-defined virtual indexes. The index advisor provides DDL scripts that you can run to create the recommended indexes. Query advisor Provides query rewrite advice based on a set of best practice rules. Access path advisor Provides advice on how to resolve access path issues that might result in poor performance.

15 Resource information in OPM System information let s you check if the system is running out of capacity shows information about memory and CPU utilization detect and isolate resource constraints Takes you to enhanced information if OPM is running in a Tivoli Enterprise Portal (TEP)

16 Reports Reporting... allows you to generate printable reports for trend analysis, capacity planning, or long-term analysis. - disk space usage - system configuration - top resource-consuming SQL statements - Database connections fully integrated into OPMs web user interface can be saved and shared as PDF, Microsoft PowerPoint, comma-separated list (CSV)

17 A few reporting examples Database Configuration Disk Space Consumption Dynamic SQL Statements

18 Extended insight: Where is my problem? Up till now focus was on database only Now it should also look beyond the database

19 Extended insight: Where is my DB application spending its time? This is what OPM measures as end-to-end (E2E) elapsed time

20 Show me the health of my apps Different Different views views at at DB DB workload. workload. Some standard applications will be auto-detected: Some standard applications will be auto-detected: SAP, Cognos, InfoSphere Warehouse, SAP, Cognos, InfoSphere Warehouse, DataStage, WebSphere DataStage, WebSphere Tells you how many DB transactions of a DB workload violated the warning or problem SLAs Tells you what the maximum E2E elapsed time for a DB transaction of a DB workload was. Tells you what the average E2E elapsed time for a DB transaction of a DB workload was and how it changed over time.

21 Built-in support for standard applications Pre-defined application specific views let you look at database workload from different angles Shows application specific terms (e.g. report name, SAP transaction code, etc.) Simply activate/de-activate the views you need for your environment WebSphere WebSphere Application Application Server Server * * WebSphere WebSphere Application Application Server Server * * Cognos Cognos FP2, FP2, or or later later SAP SAP Kernel Kernel V7.0 V7.0 SR3, SR3, or or later later InfoSphere InfoSphere Warehouse Warehouse (SQW) (SQW) Information Information Server Server *) with APAR number PK74962 *) with APAR number PK74962

22 Let me analyze what s wrong and why Where is is this DB workload spending its time client, network, or inside DB2? Most insight into DB2 will be offered with DB2 V9.7. Which statements have been executed. Which app servers or DB clients are involved. For a partition DB how does it it look on each partition. This area shows then the details to the selected layer or entity above

23 WebSphere a first class OPM citizen WebSphere support has a built-in support for OPM (starting with V6.0.21), allowing to identify problems with WAS connection 0.34 pool identify differences in the configuration of nodes in a WAS cluster check if a node in a WAS cluster has a system or network problem

24 Another addressed problem: SQL is generated EJB Query Language: 1 Query language is a subset of SQL. Doesn t have all the SQL features you want. SELECT object(e) FROM Employee e WHERE e.dept=?1 AND e.salary>=?2 OR mapping, transform to SQL 2 App query syntax is different from SQL query. How do you track problem SQL queries back to the app that issued the original query??? SQL issued to database: SELECT * FROM PROD.EMP WHERE DEPT=? AND SALARY>? 3 Often, app query is intercepted by persistence layer, and the resulting SQL query looks nothing like the app query. -> Resulting query might perform badly. -> Changing app query might not result in a similar change in the SQL query

25 How Optim purequery Runtime can help It offers three primary features: 1. Client optimization which allows converting any Java applications from dynamic to static SQL execution fix or tune problematic SQL without changing application source code Benefit from performance improvement and SQL injection prevention 2. Annotation method style to boost Java coding efficiency by offering an JPA like coding style but without the need of an heavy EJB container 3. Tracing SQL statements back to origin source (via #1 or #2) built into WAS 6.1+

26 OPM can tell you where the query came from Application source Capture SQL with purequery runtime Upload collected metadata into OPM SELECT * FROM DB2ADMIN.I SELECT * FROM DB2ADMIN.INVENTORY TestOPM my.test main 13 blahver N/P N/P blah capture

27 Product Architecture Application Monitored database Database Monitor User interface homegrown C/C++ app SAP 4 CLI driver CMX DataStage Homegrown Java app SQW 5A JCC driver DB2 on LUW V9.1+ DB2 APIs WebSphere OPM Repository & Console Server 2 3 DB2 on LUW V9.1+ HTTP(S) 1 Browser (IE/Firefox) Cognos WAS Pdq.jar 5B CMX For base product: For base product: - - Firefox Firefox or or Internet Internet Explore Explore is is required required (1). (1). - WAS can be either installed by OPM, or an - WAS can be either installed by OPM, or an existing existing WAS WAS reused reused (2). (2). - - A A DB2 DB2 V9.1+ V9.1+ is is required required to to store store performance performance information information (3). (3). A A restricted-use restricted-use DB2 DB2 license license is is shipped with OPM. shipped with OPM. For For Extended Extended Edition Edition (optional): (optional): - C/C++ applications have to upgrade the CLI - C/C++ applications have to upgrade the CLI driver (4) to a specific level. driver (4) to a specific level. - - Java Java applications applications have have to to upgrade upgrade the the JCC JCC driver driver to to a a specific specific JCC JCC level level (5A) (5A) and and to to install install a pdq.jar file (5B) into the same directory a pdq.jar file (5B) into the same directory where db2jcc.jar is stored where db2jcc.jar is stored

28 System resource monitoring IBM Tivoli Monitoring... offers various monitors for resources (network, disk storage, operating system, etc.) show real-time and historical information in Tivoli Enterprise Portal can store trend data for cpacity planning and trend analysis in Tivoli Data Warehouse allow to detect and alert staff even without being logged on can forward alerts into central troubleticket and alerting system (Tivoli Enterprise Console)

29 Tivoli and Optim system wide monitoring with drilldown ITCAM for Transactions Deep-dive tools Optim Performance Manager OPM with Extended Insight Sense Isolate Diagnose Repair Detect that a threshold has been breached and that a problem occurred, or is about to happen Pinpoint the problem to a specific part of the environment and hand-off to the appropriate specialist Drill down into the details and get to the root cause of the problem Fix the faulty component, validate the fix and roll back into production Drill down from ITCAM to OPM

30 Identify where the problem is with the help of ITCAM... Tivoli Enterprise Portal transaction topology

31 Drilling down into a DB2 problem with OPM... IBM Optim Performance Manager... shows you to which database workload this transaction belongs and how it performed in the database where is the transaction time really spent (I/O, sorting, CPU, network, etc.) if the transaction executed problematic SQL statements how an index, or MQT, or a statistic update can help

32 OPMs inflight dashboards Plugging OPM into ITCAM.. is easy: 1. Install OPM and ITCAM 2. Run installer to plug OPM screens into ITCAM 3. Configure ITCAM Data Collection 4. Done Data is NOT collected twice. OPM will feed necessary parts of its data into ITCAM!

33 Data Management Communities for DB2 IDUG the worldwide community of DB2 users Membership is FREE join today! Data Management Community share and interact with peers around the world developerworks IBM s resource for developers & IT professionals Information Champions recognizes individuals who have made the most outstanding contributions to the Information Management community

34 Beta feedback

35 Beta Program Conclusions Performance Expert was powerful, not easy to use A tool for senior and experienced DBAs; Some important info hard to find Installation complex, time consuming OPM41 is easy to use, accessible and suitable for all DBAs Installation in 30 minutes or less System health presented in 1 glance OPM41 good for time-pressured and junior DBAs Low level of skill required Easy for DBA to access from home at 3am via browser More readily available than PC-based tools (like Quest) 1 X OPM41 monitors multiple DBs, accessible anywhere OPM41 requires less skill to drive; Lower training costs, quicker ROI Installation of Extended Insight now easier Tivoli integration gives enterprise wide solution

36 Quotes from Beta 1.Drilling down through the dashboards into a problem makes DBAs more productive 2.In complex environments it can be very difficult to find the bottleneck when there is a performance problem. It can take several people with skills in application servers, networking and databases to analyse and then fix the bottleneck. This is a single tool that can handle all of that process. It is also very easy to use. 3.Installation and configuration are straightforward 4.The important information about performance from the user click rights to the database I/O subsystem is available and easily discovered through OPM 41 5.Banks need to know when there's a problem. OPM 41 presents powerful summaries, out of the box alerts and in-depth analysis that is available anywhere you can get a browser on it. For special situations, alerts can be easily customised. 6.Complex environments may have bottlenecks in several different specialist areas. This tool gives more efficient use of resources to track down and fix problems wherever they occur in these environments. 7.OPM41 makes the whole process of troubleshooting performance management issues faster and easier to understand Kaifu Chen, Senior DBA

37 Quotes from Beta contd. "I'm ready for it now" (Macy's) "Nice looking product" (Target) "This is good. In after hours monitoring, easy to say.. 'Look for something red'. Yeah. this is really good" (Pacificorp, about health summary) "This overview is what I'd ask for, read me all the numbers" (Pacificorp) "This is great. Especially getting the statement" (Pacificorp, about locking screens) "Quite promising" (Barclays)

38 Backup

39 The DB2 client information fields DB2 allows applications to send information about them to the database with each SQL operation. The database externalizes this information then in its monitoring data The performance impact of setting them is negligible (but for DB2 on LUW V9.1 FP6 is recommended) The data can be set by the application itself, or via database driver properties (see next slides) The following information can be set: Field Description Length (LUW, z/os) Client user ID Client workstation name Client application name Program name Accounting string This user ID is for identification purposes only, and is not used for any authorization. It typically identifies the user of an application. The workstation name of the client system. Some applications also use this field to identify the business transaction executed within an application. It can be used to identify the application hosted in an application server, or to identify the business transaction within an application. Identifies the application running on the client. It is only supported for a connected DB2 on z/os database. It can be used to specify charge-back information, or to add additional monitoring details about the database workload. 255, , , 32 -, , 200

40 Ways to instrument you application JDBC offers methods of class com.ibm.db2.jcc.db2basedatasource^1 public static void main(string[] args) { String url = "jdbc:db2://lap1.boeblingen.de.ibm.com:50000/demo"; Class.forName("com.ibm.db2.jcc.DB2Driver"); Connection conn = DriverManager.getConnection(url, user, password); conn.setclientinfo("clientuser", xyz"); conn.setclientinfo("clienthostname, my laptop"); } conn.preparestatement("select * FROM SYSIBM.SYSDUMMY1" + "WHERE 0 = 1").executeQuery(); CLI offers the setsqli()^2 interface SQL_API_RC SQL_API_FN sqleseti ( unsigned short DbAliasLen, char * pdbalias, unsigned short NumItems, struct sqle_client_info* pclient_info, struct sqlca * psqlca); SQL_STRUCTURE sqle_client_info { unsigned short type; unsigned short length; char *pvalue; }; ^1) see ^2) see

41 What if you can t change the application logic? For applications using the CLI driver: Specify the connection properties in file db2cli.ini^1 :... [COMMON] ; Settings for data source <data source name> [<data source name>] ClientUserID = <my end-userid> ClientApplName = <my application> ClientWrkStnName = <my system, or my business transaction> ClientAcctStr = <other charge-back or useful information>... For java applications using the JDBC driver: Specify the JDBC properties in the connection URL, such as jdbc:db2://tphkarn:50000/demo:clientworkstation=mylaptop; ^1) see ^2) see

42 More details about what EI offers for standard applications IBM WebSphere sets it with V and ifix PK74962, or V with ifix PK74962 identifies hosted application, and real user Has to be activated WAS Administrative Console -> Resources -> JDBC -> JDBC providers -> JDBC_provider. On this panel select a data source and create a new Custom Property enableendtoendmonitoringfeature of type java.lang.string and set it to TRUE. WAS Administrative Console -> Servers -> Server Types -> WebSphere application servers -> [server] -> Server Infrastructure -> Java and Process Management -> Process Definition -> Java Virtual Machine. On this panel enter the following argument into the Generic JVM Arguments field: -Dcom.ibm.cacheLocalHost=true IBM Cognos will set it with V8.4 FP2 identifies report, report package, path, and real user has to be switched explicitly on in file CQEconfig.xml under section QueryEngine (add <entry name= DB21WFM value= 1 /> ) IBM Information Server will set it with V8.5 identifies project, job, step, invocation id, and partition IBM InfoSphere Warehouse sets it with V9.7.1 identifies WH application and flow SAP sets it with SAP Kernel Version 7.0 SR3 or higher Identifies real user, SAP transaction name, SAP source module plus an additional activation via one of the following two ways: SAP profile parameter dbs/db6/dbsl_accounting = 1 SAP start environment DB6_DBSL_ACCOUNTING=1

43 How to use it purequery to capture statement information 1. Install pdq.jar and pdqmtgt.jar as discussed earlier 2. Switch on learning mode^1 via pdq.capturemode = ON Specify location of capture file via pdq.purequeryxml = <file> 3. Use PDQ Configure^2 utility to define mapping of SQL statements to DBRMs and set BIND options Optionally: change statement text 4. Use PDQ StaticBinder utility to bind DBRMs to data server 5. Switch application over to static statements ^1 via pdq.executionmode = STATIC Static mode: only previously learned and converted statements can be executed Mixed mode: previously learned and converted statements will be executed as static SQL. All others will be executed as dynamic SQL. *1) PDQ properties can be set n one of the following three ways: 1. In file pdq.properties in CLASSPATH, or in directory WEB_INF/classes (for web applications) 2. In the connection URL 3. Via method DB2SimpleDataSource.setPdqProperties() See also *2) *3)