Parallelism Strategies In The DB2 Optimizer

Size: px

Start display at page:

Download "Parallelism Strategies In The DB2 Optimizer"

Jordan Rodgers
6 years ago
Views:

1 Session: A05 Parallelism Strategies In The DB2 Optimizer Calisto Zuzarte IBM Toronto Lab May 20, :15 a.m. 10:15 a.m. Platform: DB2 on Linux, Unix and Windows The Database Partitioned Feature (DPF) or inter-partition environment and the Intra-partition environment are the two main parallelism environments in DB2. This session is about exploiting these environments to improve query performance. This presentation talks about how the DB2 optimizer works with these features that allow parallel processing for query performance. The goal is to help you understand how you can get good query performance exploiting these features that provide parallelism. 1

2 Agenda Objective Inter-Partition (DPF) Parallelism Intra-Partition (SMP) Parallelism Parallelism in the Websphere Information Integrator 2 2

Why Query Performance? Databases are getting larger Systems are getting complex Queries are getting complex Users are getting impatient SELECT prrfnbr, prnbr, pdsdesc, COALESCE((SELECT ccpr.

3 Why Query Performance? Databases are getting larger Systems are getting complex Queries are getting complex Users are getting impatient SELECT prrfnbr, prnbr, pdsdesc, COALESCE((SELECT ccpr.cppub FROM contract, cgprrel_1 AS ccpr WHERE cntrfnbr = 1 AND cntmenbr = 1 AND ccpr.cpmenbr = cntmenbr AND ccpr.cpcgnbr = mcpr.cpcgnbr AND ccpr.cpprnbr = mcpr.cpprnbr AND ccpr.cpprt = mcpr.cpprt AND ccpr.cpcatnbr = cntcatnbr), 0) AS cppub FROM cgprrel AS mcpr, product, proddesc WHERE mcpr.cpmenbr = 1 AND mcpr.cpcgnbr = AND mcpr.cpprt = '826' AND mcpr.cppub = 1 AND mcpr.cpcatnbr = 1 AND prrfnbr = mcpr.cpprnbr AND prmenbr = mcpr.cpmenbr AND pdprnbr = prrfnbr AND pdlang = 'en_gb' AND pdmenbr = prmenbr ORDER BY mcpr.cpseqnbr 3 3

4 Query Example Query the database for total sales by customers over the age of 20 by product and store SELECT C.NAME, P.NAME, ST.ADDRESS SUM(sales_amount) FROM CUST C, PROD P, SALES S, STORE ST WHERE C.AGE > 20 AND C.CUST_ID = S.CUST_ID ST.STORE_ID = S.STORE_ID P.PROD_ID = S.PROD_ID GROUP BY C.NAME, P.NAME, ST.ADDRESS 4 4

5 How Can You Help DB2 Get Good Query Performance? SELECT C.NAME, P.NAME, ST.ADDRESS SUM(sales_amount) FROM CUST C, PROD P, SALES S, STORE ST WHERE C.AGE > 20 AND C.CUST_ID = S.CUST_ID ST.STORE_ID = S.STORE_ID P.PROD_ID = S.PROD_ID GROUP BY C.NAME, P.NAME, ST.ADDRESS Indexes on the various ID Columns or C.AGE Define a Materialized Query Table Collect histogram distribution statistics on C.AGE Increase memory for the SORT or BUFFERPOOL... What about parallelism? 5 5

6 Objective : Query Performance By Improving Parallelism Many ways to help improve performance Defining indexes, Collecting Statistics, adding memory Objective of this presentation: Provide recommendations so you can improve query performance by exploiting parallelism features in DB2 6 6

7 Terminology / Acronym Quiz True or False DPF = Decimal Floating Point SMP = Symmetric multiprocessor Parallel Processing: Talking on the phone, instant messaging, watching TV and doing homework, all at the same time DPF improves performance by paralyzing SQL queries Websphere II is the version after Websphere like Viper II is the version after Viper 7 7

8 Agenda Objective Inter-Partition (DPF) Parallelism Intra-Partition (SMP) Parallelism Parallelism in the Websphere Information Integrator 8 8

9 Join Strategies Without DPF HASH JOIN HASH JOIN HASH JOIN HASH JOIN Cust HASH JOIN Cust HASH JOIN HASH JOIN Cust HASH JOIN Cust Sales Prod Prod Sales Sales Prod Prod Sales Join orders Join types H P H C S H P H S C H H P S C H H P C S Hash Joins Merge Joins Nested Loop Joins M P M C S H M P M S C H M M P S C H M M P C S H P M P M M P M P C S S C S C C S 9 9

10 Database Partitioning Feature SORT SORT SORT JOIN JOIN JOIN PROD PROD PROD CUST SALES SALES SALES STORE STORE STORE 10 10

11 Join Strategies With DPF Join strategies and partitioning concepts Collocated joins Directed joins Broadcast joins Repartitioned joins 11 11

12 Collocated Joins JOIN JOIN JOIN PROD SALES PROD SALES PROD SALES Join Predicate : P.PROD_ID = S.PROD_ID Partitioning Key : PROD : P.PROD_ID SALES : S.PROD_ID No Transfer of data between partitions 12 12

13 Directed Joins JOIN JOIN JOIN DTQ DTQ DTQ STORE SALES STORE SALES STORE SALES Join Predicate : ST.STORE_ID = S.STORE_ID Partitioning Key : STORE : ST.STORE_ID SALES : S.PROD_ID Direct specific SALES rows to appropriate partition 13 13

14 Broadcast Joins JOIN JOIN JOIN BTQ BTQ BTQ CUST SALES SALES SALES Join Predicate : C.CUST_ID = S.CUST_ID Partitioning Key : CUST : Single Node SALES : S.PROD_ID Broadcast all CUST rows to the other partitions 14 14

15 Repartitioned Joins JOIN JOIN JOIN DTQ DTQ DTQ DTQ DTQ DTQ EMP SALES EMP SALES EMP SALES Join Predicate : EMP.STORE_ID = S.STORE_ID Partitioning Key : EMPLOYES : EMP.EMP_ID SALES : S.PROD_ID SALES and EMPLOYEE rows repartitioned 15 15

16 Join Planning Strategies in DPF If a collocated join is possible done Otherwise if a directed join is possible done Otherwise choose the cheaper plan between A broadcast join and A repartitioned join H H TQ TQ S P C Many more plans considered with TQs 16 16

17 Replicated Tables JOIN JOIN JOIN BTQ BTQ BTQ CUST CUST COPY SALES CUST COPY SALES CUST COPY SALES Consider dimension tables in a star schema Avoids repeated data movement Faster query performance 17 17

18 Repartitioned Tables JOIN JOIN JOIN DTQ DTQ DTQ DTQ DTQ DTQ EMP ID EMP ST_ID SALES EMP ID EMP ST_ID SALES EMP ID EMP ST_ID SALES Consider hot columns of large table partitioned on a different column Avoids repeated data movement Faster query performance 18 18

19 Parallel Sorts Local SORT and merge at the coordinator MDTQ SORT SORT SORT SALES SALES SALES SORT after sending to the coordinator SORT DTQ SALES SALES SALES 19 19

20 Parallel Aggregation Partial aggregation during SORT Intermediate aggregation when the Grouping key covers a subset of the partitioning key Final Aggregation with global Grouping key Aggregation strategy is cost based Consider Column Group Statistics on GROUP BY columns 20 20

21 Unexpected Data Movement? Is data from your large fact table moving to a single partition dimension table? Local predicates on the fact table? For example F.C1 = 5 AND F.C2 = ABC DB2 may be underestimating the number of qualifying rows Consider replication or appropriate statistics For example column group statistics on equality predicates on fact table columns (C1, C2) 21 21

22 Fast Communication Manager (FCM) Communication is done through FCM Significantly more efficient in DB2 9 Minimizes context switching Exploits parallelism better Tuning Considerations Pay attention to FCM_NUM_BUFFERS Default of 4096 is small for typical warehouse environments recommend around "AUTOMATIC" allows some increase ~25% and drops back if not used after some time 22 22

23 Recommendations - DPF Consider collocation for the largest and frequently joined tables Consider replicated tables Consider repartitioned tables Consider appropriate statistics For example, Column Group Statistics Check FCM_NUM_BUFFERS 23 23

24 Agenda Objective Inter-Partition (DPF) Parallelism Intra-Partition (SMP) Parallelism Parallelism in the Websphere Information Integrator 24 24

25 Intra-Partition Parallelism (SMP) Exploits multiple processors on symmetric multiprocessor (SMP) architectures Could benefit a single processor system if I/O bound Combination of Data parallelism and Functional parallelism 25 25

26 Data Parallelism User not required to partition data Data dynamically assigned to query task Assigns a range of pages or rows or keys Provides dynamic load balancing Supports table and index scans Straw partitioning architecture 26 26

27 Functional Parallelism Divides query operation and assign tasks to different agent processes Single coordinator agent and multiple subagents Ensure subagents are equally busy 27 27

28 SMP Design Considerations And Concepts Optimizer design consideration Goal to reduced engineering cost Implemented as an optimizer post pass Run Time design considerations Goal to limit specialized changes Local table queues (LTQ) DEGREE 28 28

29 Optimizer Considerations SQL Query Not deeply integrated in the optimizer Main Optimizer Best Serial or DPF access plan Optimizer Post Pass SMP Parallelize Some Operators 29 29

30 SMP Access Plan RETURN (9) LTQ (8) MSJOIN (7) / \ TSCAN TSCAN (3) (6) SORT SORT (2) (5) TSCAN TSCAN (1) (4) PRODUCT PRODATR Results returned via shared memory table Queue to the coordinator agent Join processed in parallel by each sub-agent Each sub-agent scans a sort partition Hash partitioned sorts on prod_id one partition per sub-agent Parallel table scans SELECT p.name, p.prod_id, pa.attribute FROM product p, prodatr pa WHERE p.prod_id = pa.prod_id; 30 30

31 SMP Parallelism Usage In DB2 Query Operations Sorts, Joins, TEMP tables, Aggregation INSERT, DELETE, UPDATE not parallelized Utilities Index Creation, Data Load, Database Recovery 31 31

32 Recommendations - SMP Parallelism SMP parallelism is usually not recommended : In OLTP environments In concurrent multi-user environments with heavy CPU usage Recommended When CPUs are highly under utilized When DPF is not an option 32 32

33 Agenda Objective Inter-Partition (DPF) Parallelism Intra-Partition (SMP) Parallelism Parallelism in the Websphere Information Integrator 33 33

34 SMP Parallelism and Websphere II Parallelism with query parts on local data Nickname access is serialized Serialization point is the coordinator RETURN NLJN / \ TQ SHIP SCAN(T) SCAN(NN) SMP coordinator process subagent process subagent process subagent process Remote Remote DB DB RETURN NLJN / \ SHIP SCAN(NN) SCAN(T) Local data on WS II 34 34

35 DPF Federated Server Federated server separate from the DPF system No No Parallel Joins Joins Coord DPF with integrated federated server Joins Joins may may occur in in parallel Partitioned local data Fed Server Partitioned data Coord + Fed Server 35 35

36 Websphere II Trusted and Fenced Wrappers Websphere II V8.1 and earlier had trusted wrapper technology Websphere II V8.2 added fenced wrappers 36 36

37 DPF With Fenced Wrappers Optimizer has two plan choices Serial processing at the server Small amount of of data data from from the the partitioned table table Coord + Fed Server Parallel processing across appropriate partitions Large amount of of data data in in the the partitioned table table Coord + Fed Server 37 37

38 DPF Computational Partition Groups CPGs enable distribution of nickname data to partitions for parallel joins Coord + Fed server Using Using CPGs CPGs Access to to nickname is is still still serial serial but but asynchronous Coord + Fed server 38 38

39 Recommendations - Federation Make the DB2 partitioned server a federated server Use fenced wrappers enables parallelism, improve scalability, provide fault protection to the engine Use Computational Partition Groups to: Improve performance of large nickname only queries Reduce potential bottlenecks on coordinator partition Load balance the coordinator partition: Rotate the coordinator partition for federated applications Assign additional resource to a dedicated coordinator partition 39 39

40 Objective of this presentation: Provide recommendations so you can improve query performance by exploiting parallelism features in DB2 Summary Get better query performance using DPF Consider appropriate collocation Consider replicated tables Consider repartitioned tables Consider appropriate statistics If DPF is not an option, consider SMP when You have long running queries CPUs are under utilized [Not good for multi-user CPU bound systems] Improve query performance in Websphere II With DPF on the federated server Using Fenced Wrappers Using Computational Partition Groups Load balance the federation server 40 40

41 Appendix 41 41

42 DPF Example - TPCH Query SELECT nation, o_year, SUM(amount) AS sum_profit FROM (SELECT n_name as nation, year(o_orderdate) as o_year, l_extendedprice * (1 - l_discount) - ps_supplycost * l_quantity as amount FROM part, supplier, lineitem, partsupp, orders, nation WHERE s_suppkey = l_suppkey and ps_suppkey = l_suppkey and ps_partkey = l_partkey and p_partkey = l_partkey and o_orderkey = l_orderkey and s_nationkey = n_nationkey and p_name like '%coral%' ) AS profit PARTITIONING GROUP BY nation, o_year LINEITEM : L_ORDERKEY ORDER BY nation, o_year desc ORDERS : O_ORDERKEY PART : P_PARTKEY PARTSUPP : PS_PARTKEY, PS_SUPPKEY SUPPLIER : S_SUPPKEY NATION : R_REGIONKEY 42 42

43 Access Plan / \ e DTQ HSJOIN ( 8) ( 18) / \ e HSJOIN TBSCAN BTQ ( 9) ( 19) ( 20) / \ e e TBSCAN DTQ TABLE: TPCD FETCH ( 10) ( 11) SUPPLIER ( 21) / \ e e TABLE: TPCD HSJOIN IXSCAN TABLE: TPCD PARTSUPP ( 12) ( 22) NATION / \ e e TBSCAN HSJOIN INDEX: TPCD ( 13) ( 14) N_NK / \ e e e+08 TBSCAN BTQ TABLE: TPCD ORDERS ( 15) ( 16) e TABLE: TPCD TBSCAN LINEITEM ( 17) e+07 TABLE: TPCD PART Rows RETURN ( 1) 225 GRPBY ( 2) MDTQ ( 3) 225 GRPBY ( 4) 225 TBSCAN ( 5) 225 SORT ( 6) e+07 HSJOIN ( 7) / \ e DTQ HSJOIN ( 8) ( 18) 43 43

44 DPF Plan Analysis / \ E DTQ HSJOIN ( 8) ( 18) / \ e HSJOIN TBSCAN BTQ ( 9) ( 19) ( 20) / \ e e TBSCAN DTQ TABLE: FETCH ( 10) ( 11) SUPPLIER ( 21) / \ e e TABLE: TPCD HSJOIN IXSCAN TABLE: PARTSUPP ( 12) ( 22) NATION / \ e e TBSCAN HSJOIN INDEX: N_NK ( 13) ( 14) / \ e e e+08 TBSCAN BTQ TABLE: TPCD ORDERS ( 15) ( 16) e TABLE: TPCD TBSCAN Rows RETURN ( 1) 225 GRPBY ( 2) MDTQ ( 3) 225 GRPBY ( 4) 225 TBSCAN ( 5) 225 SORT ( 6) e+07 HSJOIN ( 7) / \ e DTQ HSJOIN ( 8) ( 18) LINEITEM ( 17) e+07 TABLE: TPCD PART 44 44

45 Small Table Broadcast Hash Hash Joins Joins start start with with the the build build phase. phase. HSJOIN(18) HSJOIN(18) is is part part of of the the build build phase phase of of HSJOIN(7) HSJOIN(7) NATION NATION partitioned partitioned on on REGIONKEY REGIONKEY SUPPLIER SUPPLIER partitioned partitioned on on SUPPKEY SUPPKEY Join Join predicate predicate is is on on NATIONKEY. NATIONKEY. HSJOIN ( 18) / \ TBSCAN BTQ ( 19) ( 20) TABLE: TPCD FETCH SUPPLIER ( 21) / \ IXSCAN TABLE: TPCD ( 22) NATION 25 INDEX: TPCD N_NK No No collocation collocation or or directed directed join join possible possible 45 45

46 Largest Table Filtered Early e+07 HSJOIN ( 12) / \ e e+07 TBSCAN HSJOIN ( 13) ( 14) / \ e e e+08 TABLE: TPCD TBSCAN BTQ ORDERS ( 15) ( 16) e TABLE: TPCD TBSCAN LINEITEM ( 17) e+07 TABLE: TPCD PART PART has has local local predicates Notice the the cardinality change after after the the broadcast LINEITEM is is too too huge huge to to move around!!!! 46 46

47 Biggest Join Is Collocated e+07 HSJOIN ( 12) / \ e e+07 TBSCAN HSJOIN ( 13) ( 14) / \ e e e+08 TABLE: TPCD TBSCAN BTQ ORDERS ( 15) ( 16) e TABLE: TPCD TBSCAN LINEITEM ( 17) e+07 TABLE: TPCD PART This This is is the the most most expensive join join It It is is important to to ensure that that this this join join is is collocated 47 47

48 Directed Join If No Collocation e+07 HSJOIN ( 9) / \ e e+07 TBSCAN DTQ ( 10) ( 11) e+07 TABLE: TPCD PARTSUPP ORDERS LINEITEM PART PARTSUPP: partitioned on on (PARTKEY, SUPPKEY) which are are also also the the join join columns So So a directed join join is is perfect Note Note :: Join Join result of of LINEITEM, ORDERS and and PART is is partitioned on on ORDERKEY 48 48

49 Another Directed Join e+07 HSJOIN ( 7) / \ e DTQ HSJOIN ( 8) ( 18) PARTSUPP ORDERS LINEITEM PART SUPPLIER NATION HSJOIN(7) is is based on on the the SUPPKEY column and and so so a directed join join is is good good here here Join Join result of of SUPPLIER and and NATION is is still still partitioned on on SUPPKEY Join Join result of of PARTSUPP, ORDERS, LINEITEM and and PART is is partitioned on on (PARTKEY, SUPPKEY) 49 49

50 GROUP BY And ORDER BY RETURN ( 1) 225 GRPBY ( 2) MDTQ ( 3) 225 GRPBY ( 4) 225 TBSCAN ( 5) 225 SORT ( 6) Final Final aggregation at at the the coordinator node node Rows retain order when merged at at the the coordinator Intermediate aggregation on on each each node node Rows sorted on on each each node node GROUP BY nation, o_year ORDER BY nation, o_year desc 50 50

51 Parallel Lines? 51 51

52 Session A05 Parallelism Strategies In The DB2 Optimizer Calisto Zuzarte IBM Toronto Lab 52 52

Technical Report - Distributed Database Victor FERNANDES - Université de Strasbourg /2000 TECHNICAL REPORT

Technical Report - Distributed Database Victor FERNANDES - Université de Strasbourg /2000 TECHNICAL REPORT TECHNICAL REPORT Distributed Databases And Implementation of the TPC-H Benchmark Victor FERNANDES DESS Informatique Promotion : 1999 / 2000 Page 1 / 29 TABLE OF CONTENTS ABSTRACT... 3 INTRODUCTION... 3