Best Practices. How DB2 Performance Structures Improve Performance. DB2 for z/os. Sheryl M. Larsen IBM WW DB2 for z/os Evangelist

Transcription

1 DB2 for z/os Best Practices How DB2 Performance Structures Improve Performance Sheryl M. Larsen IBM WW DB2 for z/os Evangelist

2 Sheryl M. Larsen Sheryl Larsen is an internationally recognized researcher, consultant and lecturer, specializing in DB2 and is known for her extensive expertise in SQL. She co-authored a book, DB2 Answers, Osborne-McGraw-Hill, She now works for IBM, but this material was developed before joining IBM and is the culmination of 25 years as an external consultant specializing in superhuman efforts to tune customer workloads. Her new title is IBM s World Wide DB2 for z/os Evangelist. Other titles include: President of the Midwest Database Users Group, IDUG Hall of Fame Speaker, IDUG Hall of Fame Volunteer, and Northern Illinois University Computer Science Alumni Council Board Member Sheryl has over 25 years experience in DB2, has published articles, white papers, webcasts: bmc.com, ca.com, softbase.com, ibm.developerworks.com 2

3 Disclaimer/Trademarks Copyright IBM Corporation All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS DOCUMENT HAS NOT BEEN SUBMITTED TO ANY FORMAL IBM TEST AND IS DISTRIBUTED AS IS. THE USE OF THIS INFORMATION OR THE IMPLEMENTATION OF ANY OF THESE TECHNIQUES IS A CUSTOMER RESPONSIBILITY AND DEPENDS ON THE CUSTOMER S ABILITY TO EVALUATE AND INTEGRATE THEM INTO THE CUSTOMER S OPERATIONAL ENVIRONMENT. WHILE IBM MAY HAVE REVIEWED EACH ITEM FOR ACCURACY IN A SPECIFIC SITUATION, THERE IS NO GUARANTEE THAT THE SAME OR SIMILAR RESULTS WILL BE OBTAINED ELSEWHERE. ANYONE ATTEMPTING TO ADAPT THESE TECHNIQUES TO THEIR OWN ENVIRONMENTS DO SO AT THEIR OWN RISK. ANY PERFORMANCE DATA CONTAINED IN THIS DOCUMENT WERE DETERMINED IN VARIOUS CONTROLLED LABORATORY ENVIRONMENTS AND ARE FOR REFERENCE PURPOSES ONLY. CUSTOMERS SHOULD NOT ADAPT THESE PERFORMANCE NUMBERS TO THEIR OWN ENVIRONMENTS AS SYSTEM PERFORMANCE STANDARDS. THE RESULTS THAT MAY BE OBTAINED IN OTHER OPERATING ENVIRONMENTS MAY VARY SIGNIFICANTLY. USERS OF THIS DOCUMENT SHOULD VERIFY THE APPLICABLE DATA FOR THEIR SPECIFIC ENVIRONMENT. Trademarks IBM, the IBM logo, ibm.com, and DB2 are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at Copyright and trademark information at 3

4 All the Performance Decisions Use the Right Stuff for the Job! Schema Design Partitioning Clustering Base Table Indexes Index on Expression Materialize Query Tables (MQTs) MQT Indexes ziips zaaps Accelerated Query Tables (AQTs) Appliances Performance Structures 4

5 Optimal Schema & Index & MQT Design Indexes Schema MQT MQT MQT MQT Index On Expression Indexes MQT M Q T Schema AQT 5

6 Designing Optimal Performance Structures for One Query or a Workload 6

7 One Query at a Time 1. Equal predicates first 2. Sequencing columns next 3. Predicates most restrictive to least restrictive 4. All remaining SELECT Columns C4.C5.C6.C1.C2.C3.C9.C7.C8.C12 Non-Leaf Page O O O Type 2 Index Non-Leaf Page O O Root Page O O O SELECT C7, C8, C12, C2, C3 WHERE C4 = L AND C5 = 99 AND C6 = :hv AND C1 IN (:a,:b,:c) AND C2 NOT LIKE :hv AND C3 BETWEEN :hvstart AND :hvend AND C9 = (CASE expression) Non-Leaf Page ORDER BY O C1.C2.C3 O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o 7

8 Materialized Query Tables Can be almost ANY valid SELECT statement CREATE TABLE MQT1 AS SELECT use your imagination.. ; System-maintained maintained SQL REFRESH TABLE option Deletes all rows, refills table, updates catalog User-maintained (via triggers, replication,, batch updates, etc.) REFRESH DEFERRED Via LOAD, INSERT, UPDATE, DELETE REFRESH IMMEDIATE LUW only Can be fast! 8

9 MQT. To Build or not to Build Generic MQT strategies HUGE MQT with many indexes Many small MQTs with few indexes Optimal MQT strategy Most popular queries as MQTs Common tables Common predicate sets Common data translations Indexes on the popular ones to allow access to subset of MQT 9

10 Expensive Recursive Syntax WITH ORG_CHART(BOSS, MANAGES, N) AS (SELECT BOSS, MANAGES, 1 FROM WHERE UNION ALL ORG_TABLE BOSS= GINNI SELECT T1.BOSS, T2.MANAGES, N+1 FROM WHERE ORG_CHART T2, ORG_TABLE T1 T2.MANAGES = T2.BOSS AND N < 500,000) SELECT BOSS, MANAGES, N FROM ORG_CHART Generated at Execution Ginni 10

11 MQT INDEX: BOSS MQT CREATE TABLE ORG_LIST AS ( WITH ORG_CHART(BOSS, MANAGES, N) AS (SELECT BOSS, MANAGES, 1 FROM ORG_TABLE WHERE BOSS= GINNI UNION ALL SELECT T1.BOSS, T2. MANAGES, N+1 FROM ORG_CHART T2, ORG_TABLE T1 WHERE T2. MANAGES = T2.BOSS ) SELECT BOSS, MANAGES, N FROM ORG_CHART) DATA INITIALLY DEFERRED REFRESH DEFERRED MAINTAINED BY SYSTEM ENABLE QUERY OPTMIZATION; 11

12 Query Speed Up WITH ORG_CHART (BOSS, MANAGES N) AS (SELECT BOSS, MANAGES, 1 FROM ORG_TABLE WHERE BOSS= BETH UNION ALL SELECT T1.BOSS, T2.MANAGES, N+1 FROM ORG_CHART T2, ORG_TABLE T1 WHERE T2.MANAGES = T2.BOSS AND N < 500,000) SELECT BOSS, MANAGES, N FROM ORG_CHART Accessed at Execution SELECT * FROM ORG_LIST WHERE BOSS = BETH ORG_LIST BOSS EMP N JOHN ADAM 501 BETH JIM 634 BETH SUE 634 BETH MARK 635 BETH SAM

13 Designing Indexes For a Workload 13

14 Table/Column Usage Tran tran1 tran2 c1 q1 O 1 G 1 =T4.C2 c2 O 2 G 2 =:hv c3 O 3 G 3 >800 c4 S c5 q2 s s w =22 o 1 q3 q1 O 1 O 2 O 3 c6 c7 =T2.C1 S S S =:hv o 2 O1 = Order By First Position G1 = Group By First Position S = SELECT list W = Local or Join WHERE/ON or Specify actual local/join filter Specify Group by Aggregates as: Aggr BY Column(s) c8 c9 c10 BTW 2and 7 SUM by C1.C2.C 3 AVG by C1.C2.C3 =T3.C1 s s s s s o 1 s =T3.C11 BY c11 MAX by C1.C2.C3 =T3.C1 =:hv >:hv q2 s s =: =:hv hv s s s s s O 1 IN(list) LIKE :hv: hv 14

15 tran1 DB2 for z/os Best Practices IBM Final Ranking Weight 100 Weighting Based on Importance to the Business 4200 q1 O 1 G 1 10 q2 O 2 O G 2 G W w w s s w w o q2 1 w o S w S S S 9900 w o q3 1 s w w w tran Frequency * Weight 15

16 Determining Weight Copy the frequency report Sort by business priorities Use Work Load Manager Goals for guidance Use Business Unit Owner Priorities Remove the top 30% Sort by Frequency Pull of the top number Multiply by 10 Now you have the frequency number to beat Pulls out most critical Let s say 5,500 is top frequent Then 55,000 is the goal 16

17 Grand Total By Table, by Operation Total c1 c2 c3 c4 c5 c6 c7 c8 Activity O O O S 660 O O S 4000 S 8390 G G G S 8740 S 480 W 7640 W 2020 O W 400 W 3 w O 70 6 W 3920 Use this information to justify making changes to current index design 17

18 Accelerated Query Tables for Unlimited Searches 18

19 Current Schema & Index Other Relational DBMS Index Schema RI discards 19

20 Index Design Improvements Every index except for primary and clustering was not servicing the web requests Limiting factor is number of new indexes due to expensive Q-rep Minimal index environment for B-Page searches would need 50 indexes! Optimal B-Page is unlimited search 20

21 B-Page Web Inquiry Search Attribute 1 Attribute 2 Attribute 3 Attribute 4 Attribute 5 Date 1 Date

22 Accelerated Query Table Advantage Indexxxxxx Index On Expression Indexes Schema AQT Sheryl M. Larsen, Inc

23 DB2 11 The Foundation for Business Critical Analytics OLTP Transactions Real time data ingestion High concurrency Operational analytics DB2 for z/os and IBM DB2 Analytics Accelerator DB2 Native Processing DB2 11 CPU savings benefit query workloads with or without IDAA Standard reports OLAP Complex queries 23

24 DB2 for z/os Best Practices AQTs live in the IBM DB2 Analytics Accelerator S-BladeTM IBM Dual-Core FPGA* Intel Quad-Core *FPGA - Field Programmable Gate Array IBM BladeCenter Blade 24

25 The Key to the Speed select DISTRICT, PRODUCTGRP, sum(nrx) from MTHLY_RX_TERR_DATA where MONTH = ' ' and MARKET = and SPECIALTY = 'GASTRO group by DISTRICT, PRODUCTGRP Field Programmable Gate Array FPGA Core CPU Core Slice of table MTHLY_RX_TERR_DATA (compressed) Uncompress Project Restrict, Visibility Complex Joins, Aggs, etc. sum(nrx) group by DISTRICT, PRODUCTGRP select DISTRICT, PRODUCTGRP where MONTH = ' ' and MARKET = and SPECIALTY = 'GASTRO' 25

26 Optimal ODS Search - RI Index Only & AQT Other Relational DBMS Index Schema RI discards AQT Sheryl M. Larsen, Inc

27 Best Practice Sites Have tools and staff to assist in query development and index design IBM s s Data Studio is FREE! Index design tools are extra $$,$$$ Justify IBM DB2 Analytic Accelerator for speeding up complex queries and allowing unlimited long searches Have production environments emulated in test 100% (this may not be 100% physical) 27