Investigate Database Performance the Navigator Way

Transcription

1 Dawn IBM i Client Programs Manager Scott Forstie DB2 for i Business Architect Investigate Database Performance the Navigator Way Session ID: Agenda Key: AA

2 IBM i Performance Best of the Best Focused Education Roadmap Sunday IBM i Performance Data Introduction to the IBM i Performance Data 12:45 What s New with the IBM i Performance 3:45 Today Investigate Database Performance the Navigator Way Predictive Performance 5:00 Tuesday Best Practices for IBM i Performance 11:00 Why Wait? Discover the Power of IBM i Wait 5:00 2

3 Investigate Data meets Database Brings together performance related data for system administration and database performance Need latest PTF groups, including the database group Must have the Performance Tools LPP, Manager feature, Installed IBM i 7.3 Available on IBM i 7.1 with additional enhancements in 7.2 3

4 Integration with Database Leverage the capabilities of PDI with valuable data gathered from Database Collection Services added collection of metrics SQL Plan Cache Other Database data Performance Data Investigator allows you to visualize SQL Plan Cache Snapshots and Event Monitors SQL Performance Monitor files Collection services job-level SQL metrics Navigation between database and performance tasks 4

5 Updates to the Performance Data Investigator - PTFs Major enhancements have been made to Navigator for i and the Performance Data Investigator Navigator is single source and many functions are supported on all releases IBM i 7.3! HTTP Server group PTF SF99722 Java group PTF SF99725 Database group PTF SF99723 Performance Tools group PTF SF99703 For install the latest level of: HTTP Server group PTF SF99368 Java group PTF SF99572 Database group PTF SF99701 Performance Tools group PTF SF99145 For install the latest level of: HTTP Server group PTF SF99713 Java group PTF SF99716 Database group PTF SF99702 Performance Tools group PTF SF99714 April 2017 Service Pack is the latest. 5

6 Patterns in Performance Data Performance data typically has patterns Daily, weekly, monthly, yearly Understand your typical patterns Recognize change 6

7 Introduction to Collection Services IBM i operating system function Collects system and job performance information Automated sampling technology Based on counters IBM i Performance Data Collectors 10:45 7

8 Collection Services Metrics related to Database Collection Services has been enhanced to collect more metrics related to database SQL CPU Utilization Job-level SQL metrics SQL Plan Cache Collection category of *SQL Hidden slides in your handout have more details on the above 8

9 System-Level SQL CPU time 7.2 New fields in QAPMSYSTEM file Accumulation of scaled and unscaled SQL CPU time for all jobs SYSQLCPU SYSQLSCPU Unscaled SQL CPU time used. The amount of unscaled processor time (in microseconds) spent performing work done on behalf of SQL operations. Scaled SQL CPU time used. The amount of scaled processor time (in microseconds) spent performing work done on behalf of SQL operations. 9

10 Job SQL Metrics (QAPMJOBOS) Job level activity metrics Number of SQL statements Number of SQL related database writes (logical) Number of SQL related database reads (logical) Number of SQL related database miscellaneous operations (logical) Number of SQL PAS compressions across all threads in this job Number of *SQLPKG compressions which were initiated by SQL activity in this job Connection metrics for SQL servers (Optional information that a server may or may not provide. These fields will contain blank characters if the job isn't a server, the server does not support this information, or the server is waiting for a connection to be established. One server that does provide this information is the QSQSRVR SQL server mode server) Connected job name Connected job user Connected job number Connected thread identifier

11 SQL Plan Cache New collection category *SQL New file QAPMSQLPC 7.2 (detailed metrics on hidden slides) 11

12 SQL Plan Cache SQCURQRY SQCURPLN SQPCSIZE SQPCLIMIT SQPCTHRESH SQCURSUBC SQCURMTI SQCURPRUNL SQCURROQ Total number of queries currently active Total number of plans currently in the SQL plan cache The current size (in megabytes) of the SQL plan cache The maximum size (in megabytes) that the SQL plan cache is allowed to be The maximum size (shown as a percentage of the plan cache size limit) that the SQL plan cache is allowed to be before DB2 automatically manages the SQL plan cache and replaces older plans with new plans Total number of sub caches Total number of Maintained Temporary Indexes (MTIs) Total number of pruning event listeners Total number of cached runtime objects (for queries) in the SQL plan cache New collection category *SQL New file QAPMSQLPC 12

13 SQL Plan Cache SQCURTROQ SQCURRROQ SQCURTEMP SQPLNBLT SQQRYRROQ SQQRYNRROQ SQQRYNOROQ SQPROBES Total number of cached runtime objects (for queries) in the SQL plan cache which may have retained part or all of the query answer set in a temporary copy Total number of reusable runtime objects (for queries) in the SQL plan cache Total amount of temporary storage (in megabytes) associated with the SQL plan cache Number of plans built during the interval Number of queries executed during the interval using runtime objects (for queries) where the runtime objects were candidates for being cached in the SQL plan cache for reuse by the same or other jobs Number of queries executed during the interval using runtime objects (for queries) where the runtime objects cannot be cached in the SQL plan cache Number of queries executed during the interval where a plan in the SQL plan cache was used but there were no cached runtime objects (for queries) available for use Number of times the plan cache was probed during the interval 13

14 SQL Plan Cache SQMATCH SQNOMATCH SQPCWAKE SQPCNAP SQPLNPRUN SQPLNRMV SQOPEN SQFOPEN SQFOPENROQ Number of times a plan cache probe found a match during the interval Number of times a plan cache probe was unable to find a match during the interval Number of times the plan cache woke up during the interval Number of times the plan cache fell asleep during the interval Number of plans pruned from the SQL plan cache during the interval due to plan cache size Number of obsolete plans removed from the SQL plan cache during the interval Number of full and pseudo opens that occurred during the interval Number of full opens that occurred during the interval Number of full opens that occurred during the interval which used both a plan and a cached runtime object from the SQL plan cache 14

15 SQL Plan Cache SQPOPNHCLS SQMTICRT SQMTIDLT SQAQPWAKE SQAQPRPL SQFULLOPT SQREOPT Number of queries that were hard closed during the interval Number of Maintained Temporary Indexes (MTIs) created during the interval Number of Maintained Temporary Indexes (MTIs) deleted during the interval Number of executing queries checked by Adaptive Query Processing (AQP) during the interval Number of runtime objects replaced because of Adaptive Query Processing (AQP) Number of full optimizations that occurred during the interval Number of reoptimizations that occurred during the interval when valid plans existed 15

16 Introduction to the Performance Data Investigator Collection Services collects the data The Performance Data Investigator (PDI) is how you visualize that data PDI is the Investigate Data task under Performance This presentation will review Investigate Data for Health Indicators, Collection Services, and Database Introduction to the IBM i Performance Data Investigator 12:45 16

17 Database Perspectives Select Investigate Data to use PDI tree to explore perspectives and descriptions Select a package or perspective to open a new tab and display collection list 17

18 Database tasks Use the database tasks to collect SQL performance monitor data, SQL plan cache snapshots SQL plan cache event monitors You can visualize this data using PDI 18

19 Integration with Database Cross Launch Investigate Performance Data from ACS or System i Navigator 7.1 examples 19

20 Database Perspectives 7.1 and 7.2+ Database Package SQL CPU Utilization Physical Database I/O - Detailed Logical Database I/O Detailed SQL Performance Data Collection Services Health Indicators Package (7.2 and later) Database Health Indicators

21 Database Health Indicators 7.2 and later Analyzes all collection time intervals according to the defined thresholds for database. Shows the proportion of intervals where Database health indicators exceeded the defined thresholds. Drilldowns 21

22 I/O Reads and Writes All releases 22

23 SQL CPU Utilization Overview Shows you the SQL CPU Utilization Overview system-wide perspective A starting point to determine if your CPU utilization is due to SQL or other work and later 23

24 SQL CPU Utilization by Job or Task Drill down to find the contributing jobs or tasks 24

25 Database Locks Overview All releases A graph of database record lock contention from Collection Services data 25

26 Database Locks Overview Drill down to find contributing jobs Drill down progression: (Select database record lock contention starting and ending point) Waits overview à Waits by Job or Task We can find out it was the QRWTSRVR jobs with record lock contention 26

27 Database - Collection Services Perspectives SQL Cursor & Native DB Opens Overview: q Observe the normal signature for your partition and workloads q Identify aberrations 27

28 Database - Collection Services Perspectives SQL Cursor & Native DB Opens by Job: q Zoom its easy to identify the jobs of interest 28

29 Database - Collection Services Perspectives SQL Cursor & Native DB Opens by Generic Job: q Use a macro level summarization of activity to divide & conquer 29

30 Database - Collection Services Perspectives SQL Cursor & Native DB Opens by Job Current User Profile: q Rollup database access by user q Helpful detail to solve the whodunit 30

31 Database 7.2+ Collection Services Perspectives SQL Plan Cache à Plans Detailed: q Shows the pattern of plan cache size changes q Access plan pruning could explain periodic variances in performance 31

32 Physical Database I/O - Basic and Detailed 7.1 and later Basic Provide more overview data, not broken down Two metrics charted: Physical Database I/O Reads per second Physical Database I/O Writes per second Detailed Includes breakdown by Sync/Async, SQL & Non-SQL 32

33 Physical Database I/O Basic drilldowns 33

34 Physical Database I/O Detailed Requires Job-Level Database Statistics The following metrics have been added to the job performance data *JOBMI category of Collection Services in 7.1 SQL clock time (total time in SQ and below) per thread (microseconds) SQL unscaled CPU per thread (microseconds) SQL scaled CPU per thread (microseconds) SQL synchronous database reads per thread SQL synchronous nondatabase reads per thread SQL synchronous database writes per thread SQL synchronous nondatabase writes per thread SQL asynchronous database reads per thread SQL asynchronous nondatabase reads per thread SQL asynchronous database writes per thread SQL asynchronous nondatabase writes per thread Number of high level SQL statements per thread Special instructions to activate the support i Technology Updates/page/Job Level SQL Metrics Error if you try to display one of these charts but have not activated the support: 7.1 PTFs 7.2 base OS 34

35 Physical Database I/O - Detailed 7.1 and later 35

36 Job-Level Database Statistics Ten perspectives (8 on perspective list plus 2 drilldowns) Physical Database I/O for Jobs or Tasks - Detailed Physical Database I/O for One Job or Task - Detailed 7.1 and later 36

37 Logical Database I/O - Basic and Detailed 7.1 and later Basic are the same as under Collection Services, Logical Database I/O Detailed Provide more breakdown of the data added in QAPMJOBOS Logical database I/O read, write and all other I/O requests by SQL related and non-sql related 37

38 Logical Database I/O Detailed 7.1 and later Reads, Writes & All Other I/O Requests SQL Related & Non-SQL Related Drilldown to Database I/O for One Job and Database I/O for Jobs 7 views: All I/Osfor Jobs, Reads, Writes, Others (SQL & Non-SQL) 38

39 SQL Cursor and Native DB Opens Drilldowns Charts that show SQL Cursor and Native DB Opens metrics: Native database (non-sql) Full Opens SQL Full Opens SQL Pseudo Opens as rates per second (SQL Full open count data added by Collection Services in 7.1) 39

40 SQL Cursor and Native DB Opens Native database (non-sql) File Full Opens, SQL File Full Opens, SQL File Pseudo Full Opens as rates per second Plans Built By job, generic job or job current user profile 40

41 Database Full Opens Full Opens are expensive resource-wise General recommendation is to keep Native Full Opens per second <

42 Database Full Opens In an RPG program, full opens are caused by the use of SETON *LR instead of RETRN. Avoid LR if possible. Not setting on LR (in OPM programs) will keep the program in memory, keeps file open and pointers set, retains variable values, etc. Also, avoid *NEW for the ACTGRP Shared file opens are far less expensive than full file opens. They consume less CPU, less storage and are faster than full opens. There are some implications of changing full file opens to shared file opens, but implementation of shared opens is typically easier to manage and implement than changing SETON LR to RETRN in RPG programs. The following links provide explanation, usage and considerations of using shared ODPs: Sharing database files in the same job or activation group Open considerations for files shared in a job or an activation group Input/output considerations for files shared in a job or an activation group Close considerations for files shared in a job or an activation group 42

43 SQL Performance Data Collection Services Views: 7.2 Query Opens Active Queries Plan Cache Searches plans found and plans not found Plans Detailed Maintained Temporary Indexes (MTIs) created and deleted over time Adaptive Query Processing (AQP) New Interval SQL plan cache data provided by Collection Services in 7.2 *SQL category QAPMSQLPC file 43

44 SQL Performance Data Collection Services 7.2 Query Opens Full and pseudo query opens Number of queries that were hard closed Active Query View Total number of active queries Query opens New Interval SQL plan cache data provided by Collection Services in 7.2 *SQL category QAPMSQLPC file 44

45 SQP Plan Cache SQL Overview All releases 45

46 SQL Plan Cache SQL Attribute Mix All releases 46

47 SQL Performance Monitor All releases Similar to the SQL Plan Cache perspectives, SQL Performance Monitor perspectives use PDI to graphically display SQL Performance Monitor information 47

48 Navigator what database users need to know Products IBM i Access for Windows IBM i Access Client Solutions IBM i Navigator Aliases System i Navigator or IBM i Navigator ACS Navigator for i Where does it run? Windows PC Install Anywhere Java can be used Recent service level? Best of breed features IBM i Access Windows Service Pack 7.1 SI62603 Version Schema folders and object actions, Index Advisor, and more Run SQL Scripts, SQL Performance Center VE, Schemas and more Browser Served from IBM i 7.3, 7.2, & 7.1 IBM HTTP SERVER FOR i PTF Group: SF99722 Level SF99713 Level SF99368 Level 46 PDI Perspectives Batch Modeling Historical Collection Services Data www-01.ibm.com/supp ort /docview. wss?u id=nas4sf99722 Webpage to watch www- 03.ibm.com/systems/power/software/i/acce ss / windows_sp.htm l Next (planned) Update June, ibm.com/systems/power/software/i/acce ss/solu tions_support.htm l July, 2017 (Version ) www-01.ibm.com/supp ort /docview. wss?u id=nas4sf99713 www-01.ibm.com/supp ort /docview. wss?u id=nas4sf99368 New PTF Group every 2-3 months 48

49 ACS is the focus area for DB enhancements 49

50 Case study Finding the (expensive) Ad hoc query Navigator for i on IBM i 7.2 can be used to answer tough questions: q Has my partition been operating normally? Quick, graphical assessment of health 50

51 Case study Finding the (expensive) Ad hoc query q Green is good q Threshold levels are configurable CPU (over) consumption? 51

52 Case study Finding the (expensive) Ad hoc query q Many options of where to look next Since CPU was RED, drill into the CPU details Perspectives are better than using the green screen! ü Collection Services data ü Present-time and recent past ü Easily exported ü Configurable 52

53 Case study Finding the (expensive) Ad hoc query q Quickly move through CPU Health details Follow the visual clues 53

54 Case study Finding the (expensive) Ad hoc query q Examine the data in different ways Was the spike related to a specific user? 54

55 Case study Finding the (expensive) Ad hoc query q Examine the data in different ways We identified the job(s) and the user What next? 55

56 Case study Finding the (expensive) Ad hoc query q Use System i Navigator q ACS can do the same! Query the live SQL Plan Cache for answers! 56

57 Case study Finding the (expensive) Ad hoc query q Review a subset of SQL Plan Cache detail Not concerned about light-weight queries PDI showed us the specific user name related to the CPU spikes 57

58 Case study Finding the (expensive) Ad hoc query q User, job and SQL query have been identified! Many actions: q Save to a snapshot q Visually Explain q Hold or End the job q Lower the priority q Examine the index advice 58

59 Graph History Historical performance data now part of IBM i 7.3 Configure Collection Services to turn it on and customize data retention values 59

60 60

61 & 61

62 62

63 References 63

64 IBM i Performance FAQ - a MUST read! common/ssi/ cgi-bin/ssialia s?subtype= WH &i nfotype=sa &a ppna me=st GE_PO_ PO_USE N&ht mlfi d=pow03102usen&atta chme nt=pow03102us EN. PDF 64

65 Cognitive IBM Systems i Performance on developerworks developerworks orks/i bmi/ Performance Tools i Technology Updates/page/Performance Tools Forum IBM i Performance Data Investigator IBM i Performance Data Investigator Edit Perspectives IBM i Wait Accounting How to use the Batch Model performance tool A new way to analyze historical performance data on IBM i 65

66 Special notices This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area. Information in this document concerning non-ibm products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied. All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions. IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice. IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies. All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment. Revised September 26,

67 Special notices (cont.) IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 5L, AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/os, i5/os (logo), IBM Business Partner (logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pseries, Rational, RISC System/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xseries, z/os, zseries, Active Memory, Balanced Warehouse, CacheFlow, Cool Blue, IBM Systems Director VMControl, purescale, TurboCore, Chiphopper, Cloudscape, DB2 Universal Database, DS4000, DS6000, DS8000, EnergyScale, Enterprise Workload Manager, General Parallel File System,, GPFS, HACMP, HACMP/6000, HASM, IBM Systems Director Active Energy Manager, iseries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, POWER7, System i, System p, System p5, System Storage, System z, TME 10, Workload Partitions Manager and X-Architecture are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A full list of U.S. trademarks owned by IBM may be found at: l/co pytra de.sht ml. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. AltiVec is a trademark of Freescale Semiconductor, Inc. AMD Opteron is a trademark of Advanced Micro Devices, Inc. InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries or both. Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both. NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both. SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC). The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org. TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC). UNIX is a registered trademark of The Open Group in the United States, other countries or both. Other company, product and service names may be trademarks or service marks of others. Revised December 2,