Db2 System Profile Monitoring

Transcription

1 Db2 System Profile Monitoring Denis Tronin CA Technologies Réunion du Guide Db2 pour z/os France Mardi 10 Octobre 2017 CA Tour Opus, Paris-La Défense

2 Abstract Db2 System Profile monitoring is available as of Db2 for z/os version 9, but it is still not widely used and known. However this is a quite strong Db2 capability that can help with variety of situations: manage and monitor threads and connections, model Db2 production environment, tune application behaviour. This presentation will equip you with a good understanding of Db2 profiles in general, as well as when and how to best use them CA. ALL RIGHTS RESERVED.

3 Agenda 1 DB2 SYSTEM PROFILE INTRODUCTION 2 SYSTEM MODELLING PROFILE 3 SQL OPTIMIZATION PROFILE 4 THREAD MONITORING PROFILE 5 ACCELERATOR PROFILE 6 STATEMENT SET PROFILE 7 Q&A CA. ALL RIGHTS RESERVED.

4 Db2 System Profile Introduction Overview Db2 System Profiles let you monitor and control Db2 system resources and functions within a particular context Use-cases ( Attributes / DSN_PROFILE_ATTRIBUTES) Modelling of production environment (via Explain) Changing SQL optimization system parameters Monitoring and terminating remote threads and connections Changing Db2 Accelerator thresholds Set special register and global variable values for remote applications Context ( Profile / DSN_PROFILE_TABLE) Set of filtering criteria to define the use-case scope More specific scope takes precedence Only specific criteria combinations are allowed Profile tables are created with installation job DSNTIJSG CA. ALL RIGHTS RESERVED.

5 Db2 System Profile Introduction Overview Profile definition is spread across two tables SYSIBM.DSN_PROFILE_TABLE scope SYSIBM.DSN_PROFILE_ATTRIBUTES actions SYSIBM.DSN_PROFILE_TABLE Defines a PROFILEID, its scope (AUTHID, LOCATION, PLANNAME, etc.), and whether it should be enabled during the profile start (PROFILE_ENABLED) CA. ALL RIGHTS RESERVED.

6 Db2 System Profile Introduction Overview SYSIBM.DSN_PROFILE_ATTRIBUTES Defines profile attributes that control Db2 actions when profile is activated (when criteria match) Contains the PROFILEID, KEYWORDS (= attribute ), and its value (varchar in ATTRIBUTE1, integer in ATTRIBUTE2, float in ATTRIBUTE3) History tables SYSIBM.DSN_PROFILE_HISTORY SYSIBM.DSN_PROFILE_ATTRIBUTES_HISTORY History tables Contain information about all profiles that were in effect at some point Same columns are active tables, except comment column REMARKS which is renamed to STATUS CA. ALL RIGHTS RESERVED.

7 Db2 System Profile Introduction Definition 1. Create a new profile Insert a row into SYSIBM.DSN_PROFILE_TABLE Give a unique PROFILEID Use appropriate filtering criteria: AUTHID, ROLE, LOCATION, GROUP_NAME, PRDID, PLANNAME, COLLID, PKGNAME, CLIENT_APPLNAME, CLIENT_USERID, CLIENT_WRKSTNNAME Wildcard * is allowed at the end of AUTHID & PRDID values Location IP subnet address can be specified as follows: IPv4address/mm (mm=8,16,24) example /24 IPv6/mmm (mmm=16, 32,48,64,80,96,112) example fe80:0:9d38:6ab8::/64 Set PROFILE_ENABLED ('Y is a default) Insert rows into SYSIBM.DSN_PROFILE_ATTRIBUTES Same PROFILEID, a separate row for every appropriate KEYWORD ("ACCEL_TABLE_THRESHOLD", "MONITOR THREADS", ) and its value (in columns ATTRIBUTE1, ATTRIBUTE2, or ATTRIBUTE3) CA. ALL RIGHTS RESERVED.

8 Db2 System Profile Introduction Definition 2. Start profiles with the Db2 command -START PROFILE All profile IDs with PROFILE_ENABLED=YES are processed For auto-start in Db2 12, use PROFILE_AUTOSTART=YES zparm To stop profiles, use STOP PROFILE command To check current state, use DISPLAY PROFILE command STATUS= [ON, OFF, SUSPENDED, STARTING, STOPPING] CA. ALL RIGHTS RESERVED.

9 Db2 System Profile Introduction Definition 3. Validate profile status check STATUS column in history tables SYSIBM.DSN_PROFILE_HISTORY.STATUS: REJECTED - DUPLICATED SCOPE SPECIFIED REJECTED - INVALID LOCATION SPECIFIED REJECTED - INVALID SCOPE SPECIFIED REJECTED - NO VALID RECORD FOUND IN ATTRIBUTE TABLE REJECTED - INVALID SCOPE SPECIFIED. FOR SYSTEM LEVEL MONITORING, ONLY IP ADDR, PRDID, ROLE AND/OR AUTHID, COLLECTION ID AND/OR PACKAGE NAME CAN BE SPECIFIED ACCEPTED - DOMAIN NAME IS RESOLVED INTO IP ADDRESS ACCEPTED SYSIBM.DSN_PROFILE_ATTRIBUTES_HISTORY CA. ALL RIGHTS RESERVED.

10 Db2 System Profile Introduction EXPLAIN In order to check whether a profile is used in the EXPLAIN, check REASON column in DSN_STATEMENT_TABLE explain table it would contain PROFILEID value text CA. ALL RIGHTS RESERVED.

11 CA. ALL RIGHTS RESERVED. System Modelling Profile

12 System Modelling Profile Overview Allows to model a production environment on a test system in statement EXPLAIN DSN_PROFILE_TABLE Global scope Filtering criteria columns must be NULL DSN_PROFILE_ATTRIBUTES BPname buffer pool size Use DISPLAY BUFFERPOOL to get production buffer pool size MAX_RIDBLOCKS Maximum number of RID blocks per RID pool Refer to MAXRBLK zparm in production SORT_POOL_SIZE Sort pool size Refer to SRTPOOL zparm in production CA. ALL RIGHTS RESERVED.

13 System Modelling Profile Overview System CPU performance should be modelled with zparms: SIMULATED_CPU_SPEED microseconds of TCB or SRB execution per service unit for CPU being simulated SIMULATED_CPU_COUNT number of CPU being simulated A sample SQL to get system values from production: SET CURRENT DEGREE='ANY'; EXPLAIN ALL SET QUERYNO=1234 FOR SELECT * FROM SYSIBM.SYSDUMMY1; SELECT HEX(SUBSTR(IBM_SERVICE_DATA,25,2)) AS CPU_COUNT, HEX(SUBSTR(IBM_SERVICE_DATA,69,4)) AS CPU_SPEED, HEX(SUBSTR(IBM_SERVICE_DATA,13,4)) AS RIDPOOL, HEX(SUBSTR(IBM_SERVICE_DATA,9,4)) AS SORT_POOL_SIZE FROM PLAN_TABLE WHERE QUERYNO=1234; CA. ALL RIGHTS RESERVED.

14 System Modelling Profile Scenario On production system: Run the sample SQL to capture system settings and DISPLAY BUFFERPOOL for BP sizes On test system: Create a new System Modeling profile using these settings START PROFILE Run EXPLAINs for the desired SQL workload to evaluate their future performance CA. ALL RIGHTS RESERVED.

15 System Modelling Profile Scenario On production system: o SIMULATED_CPU_COUNT = 9 o SIMULATED_CPU_SPEED = 27 o MAX_RIDBLOCKS = o SORT_POOL_SIZE = On test system: o INSERT INTO SYSIBM.DSN_PROFILE_TABLE (PROFILEID) VALUES (500) o o o INSERT INTO SYSIBM.DSN_PROFILE_ATTRIBUTES (PROFILEID, KEYWORDS, ATTRIBUTE2) VALUES (500, 'SORT_POOL_SIZE', ).. (500, 'MAX_RIDBLOCKS', 12500)..(500, 'BP0', 15000) DSNTIJUZ job: DSN6SPRM SIMULATED_CPU_COUNT=9, SIMULATED_CPU_SPEED=27.. -SET SYSPARM LOAD(xxx) EXPLAIN CA. ALL RIGHTS RESERVED.

16 CA. ALL RIGHTS RESERVED. SQL Optimization Profile

17 SQL Optimization Profile Scope Allows to set specific Db2 subsystem optimization parameters (zparms) when executing SQL statements that meet the criteria defined in the profile DSN_PROFILE_TABLE one of the following categories: AUTHID and LOCATION authorization ID and IP address (domain) PLANNAME, COLLID, and PKGNAME plan, collection, package: PLANNAME must be set to '*' COLLID collection name (can be '*') PKGNAME package name (can be '*') CA. ALL RIGHTS RESERVED.

18 SQL Optimization Profile Keywords DSN_PROFILE_ATTRIBUTES NPAGES THRESHOLD sets the maximum page threshold (NPGTHRSH zparm) to influence when Db2 should use index access (ATTRIBUTE2) -1 to use index access for as many tables as possible 0 to select the access path based on the cost (default) n to use index access on tables for which the total number of pages (NPAGES) is less than n (table statistics should be up to date) STAR JOIN enables or disables star join processing (STARJOIN zparm) ATTRIBUTE1: DISABLE, ENABLE MIN STAR JOIN TABLES sets the minimum number of tables for star join processing (SJTABLES zparm) Indicates whether to use star joins when the number of tables in the statement is equal to or greater than the specified value ATTRIBUTE2: CA. ALL RIGHTS RESERVED.

19 CA. ALL RIGHTS RESERVED. Thread Monitoring Profile

20 Thread Monitoring Profiles Overview Allows to monitor and control the use of system resources by remote threads and connections (DDF) Use to limit number of specific remote threads to keep other workload up Use to adjust time limit for specific idle DDF threads without impacting others Use to identify specific workloads CA. ALL RIGHTS RESERVED.

21 Thread Monitoring Profiles Overview DSN_PROFILE_TABLE categories depending on KEYWORDS: LOCATION only IP address (domain name), location PRDID only - product-specific ID of a monitored remote requester pppvvrrm ppp product code, vv version, rr release, m - modification product code: AQT=IDAA, ARI=Db2 VSE, DSN=Db2 for Z, JCC=JDBC & SQLJ, QSQ=Db2 on i, SQL=LUW modification: 0-4 before NFM, 5-9 NFM Example: DSN11015 is Db2 11 NFM AUTHID, ROLE, or both authorization ID and role Order of precedence as follows: Both Role only AuthID only COLLID, PKGNAME, or both Order of precedence as follows: Both Collection only Package only One of CLIENT_APPLNAME, CLIENT_USERID, CLIENT_WORKSTNNAME CA. ALL RIGHTS RESERVED.

22 Thread Monitoring Profiles Monitoring Connections DSN_PROFILE_ATTRIBUTES : MONITOR CONNECTIONS Monitors the total number of remote connections from TCP/IP application servers (including current active and inactive connections) Similar to CONDBAT zparm MAX REMOTE CONNECTED install panel field Scope: LOCATION only (IP address or domain name) ATTRIBUTE1: EXCEPTION/EXCEPTION_DIAGLEVEL1, EXCEPTION_DIAGLEVEL2, EXCEPTION_DIAGLEVEL3 new connections fail & messages issued WARNING/WARNING_DIAGLEVEL1, WARNING_ DIAGLEVEL2, WARNING_DIAGLEVEL3 only messages are issued ATTRIBUTE2: Threshold value: -1 disables check, n number of connections (can t be bigger than CONDBAT) CA. ALL RIGHTS RESERVED.

23 Thread Monitoring Profiles Monitoring Threads DSN_PROFILE_ATTRIBUTES : MONITOR THREADS Monitors the total number of concurrent active remote threads Similar to MAXDBAT zparm MAX REMOTE ACTIVE install panel field Scope: All categories ATTRIBUTE1: EXCEPTION/EXCEPTION_DIAGLEVEL1, EXCEPTION_DIAGLEVEL2, EXCEPTION_DIAGLEVEL3 new threads suspended & messages issued WARNING/WARNING_DIAGLEVEL1, WARNING_ DIAGLEVEL2, WARNING_DIAGLEVEL3 only messages are issued ATTRIBUTE2: Threshold value: -1 disables check, n number of threads (can t be bigger than MAXDBAT) CA. ALL RIGHTS RESERVED.

24 Thread Monitoring Profiles Monitoring Idle Threads DSN_PROFILE_ATTRIBUTES : MONITOR IDLE THREADS Monitors the approximate time (in seconds) that an active server thread is allowed to remain idle Similar to IDTHTOIN zparm IDLE THREAD TIMEOUT install panel field Scope: All categories ATTRIBUTE1: EXCEPTION/EXCEPTION_DIAGLEVEL1, EXCEPTION_DIAGLEVEL2 threads are aborted and messages issued EXCEPTION_ROLLBACK/EXCEPTION_ROLLBACK_DIAGLEVEL1, EXCEPTION_ROLLBACK_DIAGLEVEL2 threads aborted, messages issued, connection maintained if consistent (read-only, commit/rollback are done) WARNING,/WARNING_DIAGLEVEL1, WARNING_ DIAGLEVEL2, WARNING_MESSAGE_FOR_IDLE_TIMEOUT messages only ATTRIBUTE2: Threshold value: -1 disables check, n number of seconds CA. ALL RIGHTS RESERVED.

25 Thread Monitoring Profiles Messages DSNT771I EXCEPTION, WARNING, and DIAGLEVEL1 tells when threshold is exceeded IFCID 402 produced (if activated) DSNT772I EXCEPTION_DIAGLEVEL2 and WARNING_DIAGLEVEL2 in addition contains profile ID and reason code DSNT773I / DSNT774I WARNIING_DIAGLEVEL3, WARNING_MESSAGE_FOR_IDLE_TIMEOUT EXCEPTION_DIAGLEVEL3 in addition contains thread details CA. ALL RIGHTS RESERVED.

26 Thread Monitoring Profiles Instrumentation IFCID 402 (statistics class 4) Reports number of threads exceeded each profile warning and exception threshold type IFCID 148 (active thread) and IFCID 3 (accounting record) Reports whether a thread exceeded profile IDLE TIMEOUT threshold QWAC_PROFMON_TYPE thread monitoring profile type ( W / P ) QWAC_PROFMON_PID profile ID IFCID 406 Separate trace record providing thread details exceeded profile thresholds Alternative to messages DSNT773I / DSNT774I CA. ALL RIGHTS RESERVED.

27 Thread Monitoring Profiles Precedence summary Only one profile for a specific attribute can be applied Categories in order: 1. Location: IP address / domain name 2. Product Identifier 3. Role / Auth ID Role only Auth ID only 4. Coll ID / Package Coll ID only Package only 5. CLIENT_APPLNAME 6. CLIENT_USERID 7. CLIENT_WRKSTNNAME CA. ALL RIGHTS RESERVED.

28 Thread Monitoring Profiles Example All * zparms: CONDBAT = 2000 MAXDBAT = 500 IDTHTOIN = Suspend Aborted Db2 10! Profile: 1 LOCATION = ' /24 2 PRDID = 'DSN1001*' Active Suspend / 60 secs idle Active / 10 secs idle Attributes: 1 MONITORING CONNECTIONS EXCEPTION / MONITORING THREADS EXCEPTION / MONITORING IDLE THREADS EXCEPTION / 10 2 MONITORING THREADS WARNING / CA. ALL RIGHTS RESERVED.

29 CA. ALL RIGHTS RESERVED. Accelerator Profile

30 Accelerator Profile Threshold tuning Allows to change Db2 optimizer thresholds for selecting a query for acceleration Disclaimer contact IBM Support for details & confirmation DSN_PROFILE_TABLE Global scope Filtering criteria columns must be NULL DSN_PROFILE_ATTRIBUTES ACCEL_TABLE_THRESHOLD sets the minimum total table cardinality for a query to be accelerated: ATTRIBUTE2: -1 disables the check, n for number of rows ACCEL_RESULTSIZE_THRESHOLD sets the maximum number of rows to return ATTRIBUTE2: -1 disables the check, n thousands of rows ACCEL_TOTALCOST_THRESHOLD sets minimum estimated total cost ATTRIBUTE3: -1.0 disables the check, n.n - cost CA. ALL RIGHTS RESERVED.

31 Accelerator Profile Eligibility analysis Enables EXPLAIN to evaluate dynamic SQL query's eligibility for a specific virtual accelerator Applies to dynamic EXPLAIN and CURRENT EXPLAIN MODE = EXPLAIN DSN_PROFILE_TABLE any of the following categories Global scope Filtering criteria columns must be NULL AUTHID and LOCATION authorization ID and IP address (domain) PLANNAME, COLLID, and PKGNAME plan, collection, package DSN_PROFILE_ATTRIBUTES ACCEL_NAME_EXPLAIN specifies that a dynamic EXPLAIN statement will evaluate a dynamic SQL query for acceleration to the accelerator that is defined in ATTRIBUTE1 For default analysis, it might be easier to set the modelling zparm: ACCELMODEL=YES CA. ALL RIGHTS RESERVED.

32 Accelerator Profile Eligibility analysis Instrumentation Accounting record IFCIDs 3 and 148: QWAC_ACCEL_ELIG_ELA eligible elapsed time QWAC_ACCEL_ELIG_CP eligible CP time QWAC_ACCEL_ELIG_SE eligible ziip time Dynamic SQL statistics IFCID 316: QW0316FL2 (x 80 - QW0316ELI) accelerator eligibility flag Static SQL statistics IFCID 401: QW0401FL (x 80 - QW0401ELI) accelerator eligibility flag CA. ALL RIGHTS RESERVED.

33 CA. ALL RIGHTS RESERVED. Statement SET Profile

34 Statement SET Profile Overview Allows setting the value of a special register or build-in global variables Applies to the remote applications only DSN_PROFILE_TABLE any of the following categories (Similarly to Thread Monitoring profile) LOCATION only IP address or domain name PRDID only - product-specific ID of a monitored remote requester AUTHID, ROLE, or both authorization ID and role Order of precedence as follows: Both Role only AuthID only COLLID, PKGNAME, or both Order of precedence as follows: Both Collection only Package only One of CLIENT_APPLNAME, CLIENT_USERID, CLIENT_WORKSTNNAME CA. ALL RIGHTS RESERVED.

35 Statement SET Profile Details DSN_PROFILE_ATTRIBUTES SPECIAL_REGISTER SET statement text provided in ATTRIBUTE1: SET CURRENT APPLICATION COMPATIBILITY =.. SET CURRENT DEGREE =.. SET CURRENT EXPLAIN MODE =.. SET CURRENT QUERY ACCELERATION =.. SET CURRENT SCHEMA =.. SET CURRENT TEMPORAL BUSINESS_TIME =.. and more! (25 registers in total) GLOBAL_VARIABLE SET statement in ATTRIBUTE1 column: SET SYSIBMADM.GET_ARCHIVE =.. SET SYSIBMADM.MOVE_TO_ARCHIVE =.. SET SYSIBM.TEMPORAL_LOGICAL_TRANSACTIONS =.. SET SYSIBM.TEMPORAL_LOGICAL_TRANSACTION_TIME =.. Db CA. ALL RIGHTS RESERVED.

36 Statement SET Profile Instrumentation IFCID 387: Traces values set by the SET profile Recorded values are from the time they are applied to a connection Reported fields are in the repeating groups, one per register/variable Example to enable trace collection: STA TRACE(MON) CLASS(30) IFCID(387) DEST(SMF) Sample report output: CA. ALL RIGHTS RESERVED.

37 CA. ALL RIGHTS RESERVED. Q&A

38 References Db2 12 Managing Performance Db2 11 Subsystem and Transaction Monitoring and Tuning CA. ALL RIGHTS RESERVED.

39 Db2 System Profile Monitoring Db2 System Profile Monitoring Denis Tronin