Tuning DB2 to Reduce Your Rolling 4 Hour Average Costs

Transcription

1 Tuning DB2 to Reduce Your Rolling 4 Hour Average Costs align Craig S. Mullins Mullins Consulting, Inc. Sponsored by: 2012 Mullins Consulting, Inc.

2 Author This presentation was prepared by: Craig S. Mullins President & Principal Consultant Mullins Consulting, Inc. 15 Coventry Ct Sugar Land, TX Tel: Fax: Now Available: 6 th edition covering up thru DB2 10 for z/os This document is protected under the copyright laws of the United States and other countries as an unpublished work. Any use or disclosure in whole or in part of this information without the express written permission of Mullins Consulting, Inc. is prohibited Craig S. Mullins, Mullins Consulting, Inc. All rights reserved Mullins Consulting, Inc. 2

3 Agenda Pricing, Reducing Cost, and Tuning DB2 for z/os General Terminology Pricing models VWLC SubCapacity Report Tool (SCRT) Understanding IMSU and R4H But I thought it was usage pricing? What does this mean for DB2? DB2 tuning approaches to reduce the R4H and therefore reduce your monthly software bill Mullins Consulting, Inc. 3

4 Terminology Technical terms & abbreviations used in this presentation. MSU: (Million Service Units) : reference unit for Software billing. IMSU (or ACTMSU): instantaneous MSU consumption R4H: Rolling 4 Hours DC: Defined Capacity CPC and CEC CPC: Central Processor Complex CEC: Central Electronic Complex CPC Capacity: z/server full capacity HMC: Hardware Management Console 2012 Mullins Consulting, Inc. 4

5 MIPS vs. MSUs MIPS: an acronym for Millions of Instructions Per Second. No longer in vogue MSU: million service units is a measurement of the amount of processing work that can be performed in an hour. One service unit originally related to an actual hardware performance measurement, but that is no longer the case. A service unit is an imprecise measurement E.g., tubes of toothpaste. IBM publishes MSU ratings for every mainframe model Mullins Consulting, Inc. 5

6 Licensing vs. Pricing It is important to understand the difference between licensing and pricing. If you execute IBM zseries software on a CPC, you must have a license to do so. A license for an IBM monthly license charge product is specific to the product and a CPC with a particular serial number. The license is sold in terms of MSUs. If you are executing a product on a 1500 MSU CPC, you must have a 1500 MSU license, specifying the serial number of that CPC. The price for a product, that is, how much you pay IBM each month, depends on the pricing metric that is used for that product. Examples of pricing metrics include PSLC and VWLC Mullins Consulting, Inc. 6

7 MLC Products Monthly License Charge (MLC) Products Monthly license charges apply to many IBM software products Some of the most common include z/os, DB2, CICS, IMS, MQSeries, and COBOL There are others. Pricing and terms and conditions for MLC products are based on the pricing metric you select. Pricing metrics can roughly be grouped into two categories: Full capacity Sub-capacity 2012 Mullins Consulting, Inc. 7

8 Full vs. Sub Capacity Full capacity based pricing metrics Under a full capacity-based metric, all software charges are determined by the capacity of the CPC in which the product runs. Parallel Sysplex License Charges (PSLC) and zseries Entry License Charges (zelc) are examples of full capacity based metrics. Sub-capacity capable pricing metrics Under a sub-capacity metric, software charges for certain products are based on the utilization capacity of the LPARs in which the product runs. Workload License Charges (WLC) and Entry Workload License Charges (EWLC) are examples of sub-capacity capable pricing metrics Mullins Consulting, Inc. 8

9 IBM Software Pricing Metrics * Metric eligible for Sub Capacity charges + Metric eligible for Aggregation in a qualified Parallel Sysplex 1 Customer may select VWLC pricing for a z196 machine only when the z196 participates in a VWLC Sysplex under the AWLC Transition Charges for Sysplexes terms and conditions with at least one of the following machines (z10 EC, z10 BC, z9 EC, z9 BC). 2 VWLC and FWLC are only available on z10 BC, z9 BC, z890 or z800 when that machine is participating in a qualified Parallel Sysplex environment. 3 PSLC are only available on z10 BC, z9 BC, z890 or z800 when that machine is participating in a qualified Parallel Sysplex environment. 4 The z10 BC model A01, z9 BC model A01, and the z890 model 110 are priced using zseries Entry License Charges (zelc) Mullins Consulting, Inc. 9

10 Sub-Capacity Pricing Advantages Your charges for products that use sub-capacity pricing are based on how much the LPARs in which the products run utilize system resources, rather than on the full capacity of the CPC. You can purchase hardware capacity for future needs without incurring an immediate increase in your software bill. If your usage decreases when business is slow, your software bill decreases with it. If your usage is seasonal, your monthly software bills are lower during periods of lower usage. You pay for capacity on a rolling four-hour (R4H) average, not on maximum capacity reached Mullins Consulting, Inc. 10

11 Be Aware Sub-capacity pricing can, in some cases, lower your overall software bill while increasing some components of your bill. Perhaps some of your software products have higher costs with sub-capacity pricing, but the overall software bill is lower. Or, maybe you have a seasonal usage pattern, so your bill might be higher in some months with sub-capacity pricing, but your total annual bill for all eligible products is lower Mullins Consulting, Inc. 11

12 Moving to VWLC If you are not already using subcapacity pricing Read Planning for Subcapacity Pricing (A ) Step by Step Guide Form a team Prerequisites Timeline Software inventory Run a planning tool Get a cost analysis from IBM Create cost chart Review T&C s 2012 Mullins Consulting, Inc. 12

13 The 4 parts of VWLC! 2012 Mullins Consulting, Inc. 13

14 Part 1 of VWLC R4H 2012 Mullins Consulting, Inc. 14

15 VWLC The R4H The R4H represents the average consumption (in MSU) of the LPAR during the last 4 hours = Average of the last 48 IMSU metrics. 9:00 10:00. 11:00 1: R4H :0 0 19:00 20:00 23: R4H LPAR :00 9:00 12:00 15:00 18:00 21:00 0:00 3:00 6:00 IMSU LPAR 1 R4H LPAR Mullins Consulting, Inc. 15

16 Clarifying R4H Every 5 minutes IMSU consumption is measured The R4H is an average of the past 48 IMSU metrics 48 (metrics) / 4 (hours) = 12 metrics/hour minutes added to the calculation 119 removed from the calculation 2012 Mullins Consulting, Inc. 16

17 How Does R4H Translate Into Billing? VWLC is not a product usage based pricing; instead, it is a bit of a hybrid. Remember we use IMSU to calculate R4H. So, assume you have three LPARs. R4H averages are calculated for each qualified product, each hour, for each LPAR, for the month. A 30 day month has 720 hours (24 x 30 = 720) Hour LPAR1 LPAR2 LPAR Mullins Consulting, Inc. 17

18 How Does R4H Translate Into Billing? (continued) Now we also have various products in each LPAR LPAR1: z/os, DB2, MQ LPAR2: z/os, CICS, DB2, COBOL LPAR3: z/os, CICS, IMS, COBOL The products will be charged based on the peak period of the sum of the R4H for the LPARs in which they run z/os: LPAR1, LPAR2, LPAR3 CICS: LPAR2, LPAR3 DB2: LPAR1, LPAR2 MQ: LPAR1 COBOL: LPAR2, LPAR3 IMS: LPAR Mullins Consulting, Inc. 18

19 How Does R4H Translate Into Billing? (continued) We need R4H sums for all combination with products Hour MQ IMS LPAR1 LPAR2 LPAR z/os DB CICS, COBOL Mullins Consulting, Inc. 19

20 How Does R4H Translate Into Billing? (continued) Q: So what is the billable MSU for each product? A: It is the peak R4H for the LPAR or grouping of LPARs in which the product ran. Hour LPAR1 MQ = 60 LPAR3 IMS = z/os = DB2 = CICS, COBOL = Mullins Consulting, Inc. 20

21 Part 2 of VWLC DC DC 2012 Mullins Consulting, Inc. 21

22 DC vs. WLM vs. PR/SM Defined Capacity (DC) Set in the HMC and used to control billing Does not enforce capping and is not mandatory z/os Workload Manager (WLM) Calculates and monitors the R4H If DC is set to a non-zero number, WLM monitors the R4H and ensures that the R4H is less than or equal to the DC Of course, WLM also manages your workloads based on settings/goals (WLM operations are not within scope of this course) Processor Resource/Systems Manager (PR/SM) Enforces the soft cap when WLM determines it is needed 2012 Mullins Consulting, Inc. 22

23 Defined Capacity Hardware Management Console (HMC) you set the DC here 2012 Mullins Consulting, Inc. 23

24 Part 3 of VWLC IMSU 2012 Mullins Consulting, Inc. 24

25 IMSU IMSU : Instantaneous consumption of MSU for the LPAR 10:00 Peak of consumption AM :00 Peak of consumption PM :00 14:00 Lunch time :00 Beginning of OLTP 21:00 Beginning of Batch :00 06:00 12:00 18:00 00: Mullins Consulting, Inc. 25

26 Part 4 of VWLC SCRT 2012 Mullins Consulting, Inc. 26

27 VWLC IBM Sub Capacity Reporting Your Invoice ==B5========= SCRT SUB-CAPACITY REPORT - IBM Corp ==================== Run Date/Time 05 Mar :55 Name of Person Submitting Report: Joe User Address of Report Submitter: joe.user@softwareonz.com Phone Number of Report Submitter: Customer Name SOFTWAREONZ Section P5 Customer Number 216 Machine Serial Number EE Machine Type and Model Machine Rated Capacity (MSUs) 422 Purchase Order Number (optional) Customer Comments (255 chars max) (optional) For recurring charge (MLC) products the data supplied in this report will be used to adjust the billable MSUs in inventory for all MLC Products listed under the MLC Product Name column on this report. In accordance with our agreement IBM will treat a change in product licensed capacity as an order. If the MSUs have changed since the last report software billing based on inventory MSUs will increase or decrease accordingly. For One Time Charge (IPLA) products the data supplied in this report will be used to bill those IPLA products listed under the IPLA Product Name column in this report which exceed your entitled capacity. In accordance with our agreement IBM will treat the use of a product in excess of its entitled capacity as an order and you will be billed for the amount in excess of your entitlement. Note: This report is expected to provide a "% data collected" > 95% and data reporting period beginning on the 2nd of the previous month and ending on the 1st of the current month. ==C5=============================================================== 2012 Mullins Consulting, Inc. 27

28 What is the SCRT? The Sub-Capacity Reporting Tool (SCRT) is a no-charge IBM tool that reports required license capacity for sub-capacity eligible products. The SCRT indicates the required license capacity (in MSUs) of each sub-capacity eligible product. The SCRT cross-references LPAR utilization and product execution by LPAR to determine the maximum concurrent LPAR four-hour rolling average utilization the highest combined utilization of LPARs where each product executes during the reporting period. Sub-capacity products are charged based on the rolling fourhour (R4H) average utilization of the LPARs in which the subcapacity products execute. The sub-capacity report determines the required license capacity by examining, for each hour in the reporting period: The four-hour rolling average utilization, by LPAR Which eligible products were active in each LPAR 2012 Mullins Consulting, Inc. 28

29 R4H or DC: What is Reported? Each hour, the (R4H) rolling four-hour average is compared to the defined capacity (DC), if set. If the DC is set, the SCRT uses the lower of the two values for the utilization value for the z/os system for that hour. If R4H < DC then R4H is used If R4H > DC then DC is used If the DC is set to zero (that is, no soft capping), the SCRT uses the R4H as the utilization value for the z/os system for that hour. Remember, the R4H is the average value of SMF 70 for that hour Mullins Consulting, Inc. 29

30 IBM Sub Capacity Reporting Tool (SCRT) SCRT (Sub Capacity Reporting Tool) is a tool used by IBM for z/os billing. It uses the following SMF records: SMF 70-1 and SMF 89-1 / 89-2 The SCRT is computed on a monthly basis : During the month (from the second day of the month at 0h00 to the first day at 24h00 of the following month) R4H averages are calculated for each WLC products, each hour, for each LPAR, for the month Mullins Consulting, Inc. 30

31 The Soft Capping Rule When Rolling 4 Hours (R4H) becomes greater than or equal to DC then the LPAR is capped. That means that the IMSU consumption will not be able to exceed DC anymore until the R4H becomes lower than the DC Mullins Consulting, Inc. 31

32 VWLC with Soft-Capping: LPAR view IMSU : Instantaneous consumption of MSU for the LPAR 10:00 Peak of consumption AM 17:00 Peak of consumption PM :00 14: Lunch time : :00 Beginning of OLTP Beginning of Batch :00 06:00 12:00 18:00 00: Mullins Consulting, Inc. 32

33 VWLC with Soft-Capping: LPAR view IMSU : Instantaneous consumption of MSU for the LPAR R4H : Average of IMSU in 4 consecutive hours. and that you can fix at any level you want :00 06:00 12:00 18:00 00: Mullins Consulting, Inc. 33

34 VWLC with Soft-Capping: LPAR view IMSU : Instantaneous consumption of MSU for the LPAR R4H : Average of IMSU in 4 consecutive hours. DC : Defined Capacity, billing limit that you don t want to exceed and that you can fix at any level you want But performance can be affected (capping IMSU is brought back to the DC)! 0 00:00 06:00 12:00 18:00 00: Mullins Consulting, Inc. 34

35 Conclusions Pricing models and calculations can be confusing but Subcapacity pricing can save you money! There are more nuances to VWLC than can be adequately covered in an introductory presenation oocod, GCL, etc. It is possible to impact the bill for multiple products by reducing the R4H for a single product. Subcapacity pricing charges by LPAR usage, so if the R4H for an LPAR decreases, it is possible that the bill for every product in that LPAR will go down. Intelligent DB2 tuning can be helpful! 2012 Mullins Consulting, Inc. 35

36 Tactics for DB2 Tuning Understand where your monthly peaks are and attack the peaks. The SCRT reports the peak for each LPAR As well as the next highest peak usage time When DB2 contributes to the peak, use DB2 tuning techniques to reduce consumption When you reduce one peak, find the next and attack that one. Let s look at some techniques that can be used Mullins Consulting, Inc. 36

37 Tuning Outside the Box Remember, you are charged for each product based upon the LPARs in which the product runs. You may be able to tune DB2 usage and reduce the cost of other products that run in the same LPARs z/os, CICS, IMS, COBOL, etc. Likewise, you may be able to tune non-db2 usage and reduce the cost of DB2 Consider: say you have a batch VSAM program that consumes a lot of resources during your monthly peak, tuning that program to be more efficient could reduce your peak R4H, and thereby reduce the cost for all products running in that LPAR. For example, running a REPRO to reorganize the file and eliminate CI and CA splits can improve VSAM performance and reduce resource requirements And if that job ran in the same LPAR as DB2, it could also reduce the cost of DB Mullins Consulting, Inc. 37

38 Where is Your Peak? What if your monthly R4H peak is in the batch window? Are you meeting your batch SLAs? Consider what can be done to reduce MSU consumption during batch processing. COMMIT processing? Concurrency and locking? SQL tuning? What if your monthly R4H peak is during your daily transaction processing? Have your optimized your transactions? Is the workload appropriate for your system? Consider what can be done to reduce MSU consumption during OLTP. CICS tuning? RDO specifications? SQL tuning? 2012 Mullins Consulting, Inc. 38

39 DB2 Tuning Tactics Tune the SQL/application Most relational tuning experts agree that the majority of performance problems with relational database applications are caused by poorly coded application code and SQL. Other DB2 tuning options Tune the database Tune the subsystem Tune the environment Mullins Consulting, Inc. 39

40 SQL/Application Tuning Identify the heaviest resource consuming SQL during the peak processing time(s) Identify tuning opportunities Review access paths and tune Indexing, eliminating scans Let SQL do the work (not the program) Avoid sorting when possible Favor Stage 1 and Indexable predicates Review locking considerations UOW, lock avoidance, COMMIT, isolation level (dirty read) Review cursors Multi-row FETCH 2012 Mullins Consulting, Inc. 40

41 Let s Take a Look at Multi-row Fetch With multi-row Fetch, instead of obtaining data a row at a time to the program, data is obtained in blocks That is, multiple rows are obtained into an array in the program IF-THEN-ELSE The program then must navigate through the array and request an additional block of rows when the current block has been processed Let s look at high-level depictions of how a multi-row FETCH differs from a traditional, single row FETCH cursor 2012 Mullins Consulting, Inc. 41

42 Typical DB2 Single-Row FETCH Program DB2 Result Set Start FETCH 1 Row FETCH 1 Row FETCH 1 Row Request Data Request Data Request Data Row 1 Row 2 Row Mullins Consulting, Inc. 42

43 Example DB2 Multi-Row FETCH Program Start FETCH 10 Rows FETCH 10 Rows Request Data Request Data DB2 Result Set Row 1 Row 2 Row 3 Row 4 Row 5 Row 6 Row 7 Row 8 Row 9 Row 10 Row 11 Row 12 Row 13 Row 14 Row 15 Row 16 Row 17 Row 18 Row 19 Row 20 Row Mullins Consulting, Inc. 43

44 Single Row vs. Multi-Row FETCH With Multi-Row FETCH: CPU can be greatly reduced because moving data across address spaces is expensive With a multi-row FETCH multiple rows are moved with one operation With a row-at-a-time FETCH, each FETCH requires an operation More coding is required in the application program to enable Unless it is a distributed program using Block Fetch Block Fetch is a distributed technique for more efficiently blocking fetched rows; when data is blocked, less overhead is required as data is sent over the communication lines As of DB2 V8, Block Fetch will automatically use multi-row FETCH To ensure that your distributed read only cursors use Block Fetch be sure to code the FOR READ ONLY clause and BIND specifying CURRENTDATA NO Mullins Consulting, Inc. 44

45 Performance of Multi-Row FETCH The performance improvement using multi-row FETCH in general depends on: Whether or not the request is distributed Number of rows fetched in one FETCH Number of columns fetched, data type and size of the columns In general, the fewer columns the greater the performance improvement with multi-row FETCH will be Complexity of the SELECT processing The fixed overhead saved for not having to go between the database engine and the application program has a lower percentage impact for complex SQL that has longer path lengths 2012 Mullins Consulting, Inc. 45

46 General Advice Use Multi-row FETCH to improve application performance in all new development efforts Use it whenever 10 or more rows need to be fetched Consider retrofitting it into existing application programs Many times new features are only used on new programs instead of examining existing workload to determine where the new features can improve performance MRF should be a prime candidate for retrofitting into existing applications Look for programs that contain SQL statements from which a lot of data is fetched 2012 Mullins Consulting, Inc. 46

47 Use SoftBase Attach Facility! SoftBase Attach Facility s MRF Feature Quickly implement MRF in your existing applications SQL statement CPU and elapsed time savings as high as 74%* No changes to source code Automatically implement MRF *Based on observations from internal testing of individual SQL statements. Individual results may vary, and will be based on many factors, including the number of FETCH statements an application initiates Mullins Consulting, Inc. 47

48 SoftBase Attach Faciltiy MRF Feature Program Attach Facility MRF Feature DB2 Result Set Start FETCH 1 Row FETCH 1 Row FETCH 1 Row Request 1 Intercept single-row FETCH call 4 Return rows as they are called FAST! 2 Issue Multi-Row FETCH call 3 Pull rows into temporary result set 5 After all rows are called, the process repeats. Request Row 1 Row 2 Row 3 Row 4 Row 5 Row 6 Row 7 Row 8 Row 9 Row 10 Row 11 Row 12 Row 13 Row 14 Row 15 Row 16 Row 17 Row 18 Row 19 Row 20 Row Mullins Consulting, Inc. 48

49 Attach Facility s Multi-Row FETCH Feature How does it save CPU time? Each row can be pulled much faster Each FETCH requires only tens of instructions, versus hundreds or thousands when issued to DB2 Tests have shown FETCH processing performance improvement by as much as 74%* *Results may vary MRF Feature FETCH Processing Program FETCH single row Program FETCH single row 10 s of instructions per fetch MRF Temporary Table Row 1 Row 2 Row 3 Row 4 Row 5 Typical DB2 FETCH Processing DB2 Result Set Row 1 Row 2 Row 3 Row 4 Row s 100 s or or 1000 s 1000 s of instructions of instructions per fetch 2012 Mullins Consulting, Inc. 49

50 Attach Facility MRF Feature 2012 Mullins Consulting, Inc. 50

51 Attach Facility MRF Feature User Site Results CPU Time Reduction Elapsed Time Reduction Description 75% 75% buffer_size=100 rows, Fetch:Open 100:0 (i.e very high) 67% 67% N/A 65% 58% N/A 46% No data buffer_size=100 rows, Fetch:Open 97:3 45% 35% N/A 43% No data buffer_size=100 rows, Fetch:Open 97:3 28% 9% buffer_size=100 rows, Fetch:Open - 81:19, ORDER BY was used 24% 43% buffer_size=100 rows, Fetch:Open 97:3 23% 48% N/A 2012 Mullins Consulting, Inc. 51

52 Where Can MRF Help? SoftBase s Batch Analyzer quickly identifies applications that can benefit from MRF Download and install SoftBase s Batch Analyzer free trial Run for several days (very low overhead) Output identifies applications that will benefit the most from multi-row FETCH JOB NAME JOB NUMBER PROGRAM NAME CURSOR NAME NUMBER OPENS NUMBER FETCHES AVERAGE FET/OPEN OPEN DB2 ELAPSED TIME OPEN DB2 CPU TIME FET DB2 ELAPSED TIME FET DB2 CPU TIME ABC7LOTS JOB01482 PRGM3 C : : :43.2 1:00:28 ABC7LOTS JOB01482 PRGM3 C : : :42.8 1:01:35 ABC7LOTS JOB01482 PRGM3 C : :00.1 1:01:36 1:03:34 ABC7LOTS JOB01473 PRGM1 C : : : :28.8 ABC7LOTS JOB01473 PRGM1 C : : : :30.4 ABC7LOTS JOB01473 PRGM1 C : : : :39.4 ABC7RUN JOB08479 PRGM2 GET_JOBS : : : :00.0 ABC7RUN JOB08480 PRGM2 GET_PROG : : : :00.0 ABC7RUN JOB08481 PRGM2 PACKSTMT : : : : Mullins Consulting, Inc. 52

53 SoftBase Batch Analyzer Drill down reporting lets you quickly find: Worst Jobs Worst Job Steps Worst SQL Statements Individual SQL Statement Text Hot key opens DB/IQ and explains the SQL problem! 2012 Mullins Consulting, Inc. 53

54 DB/IQ and Batch Analyzer Improve SQL Quality and Development Efficiency; Reduce Costs DB/IQ Quality Assurance Enables DBAs to automatically enforce SQL standards, resulting in improved SQL quality and reduced CPU consumption Reduces development costs by enabling faster, better coding Batch Analyzer Enables programmers to quickly find and fix poorly performing SQL Reduces time spent troubleshooting and, ultimately, mainframe usage expenses 2012 Mullins Consulting, Inc. 54

55 Summary Consider cost reduction along with performance improvement by taking peak R4H into account Find the big resource consumers in your peak R4H window Look for ways to minimize resource consumption Find areas where MRF can improve DB2 performance Consider a proactive approach with SoftBase tools: Attach Facility Batch Analyzer DB/IQ 2012 Mullins Consulting, Inc. 55

56 Contact Information Craig S. Mullins Mullins Consulting, Inc. 15 Coventry Court Sugar Land, TX Phone: (281) th edition now available covering up thru DB2 V10! Mullins Consulting, Inc. 56

57 SoftBase Contact Information SoftBase 20 Fall Pippin Lane, Suite 202 Asheville, NC USA Mullins Consulting, Inc. 57 SoftBase is a division of Candescent SoftBase, LLC.