Advanced Design Considerations

Transcription

1 Advanced Design Considerations par Phil Grainger, BMC Réunion du Guide DB2 pour z/os France Mercredi 25 novembre 2015 Hôtel Hilton CNIT, Paris-La Défense

2 Introduction Over the last few years, we have gained many new abilities with DB2 Sometimes, people jump on new things without thinking Sometimes people are scared of new Let s look how some of them work And consider some thoughts on their usage 2

3 Introduction In alphabetical order, we have: Avoiding Locked Data DPSIs Hash Access Index Include Index on Expression Inline LOBs Instead Of Triggers Materialised Query Tables Partition by Growth Partition Rotation Random Indexes Table Clones Temporal Data Truncate statement XML Native Storage And others not included 3

4 Avoiding Locked Data 4

5 Avoiding Locked Data Trying to access locked rows used to have a simple result You waited for the lock to be released Or you timed out Or were deadlocked Then we gained ISOLATION (UR) Uncommitted read This read data even when it is locked Then we gained Skip Locked Data This skipped over any locked rows/pages And just returned the unlocked data Followed by Last Committed Return the locked data as of last commit Only for Insert and Delete though 5

6 Avoiding Locked Data Which is best? It depends on what you are trying to achieve And what the application can tolerate in the way of data inconsistencies There is no single right answer But don t choose the WRONG option, the results are all subtly different 6

7 Data Partitioned Secondary Indexes DPSIs 7

8 Table Based Partitioning A partitioned table NO LONGER needs a partitioning index The table itself is defined as partitioned with the relevant key ranges Note that the tablespace in which the table will reside must also have the same number of partitions CREATE TABLE table_name (col1, col2, col3), PARTITION BY (col1, col2), ( PART 1 VALUES (aaa,xxx), PART n VALUES (yyy,zzz)); Or PARTITION n ENDING AT (aaa,xxx) 8

9 Table Based Partitioning Version 8 partitioned table NOTE No partitioning index Still partitioned by customer number, and clustered likewise 101 IL 102 AK 103 AK 104 OR 105 IL 201 AK 202 LA 203 NV 204 IL 205 OR 301 AK 302 IL 303 NV 304 AK 305 TX

10 Table Based Partitioning Version 8 partitioned table With a partitioned index on customer Still partitioned by customer number, now clustered by state! AK 102 AK 103 IL 101 IL 105 OR 104 AK 201 IL 204 LA 202 NV 203 OR 205 AK 301 AK 304 IL 302 NV 303 TX 305 DPSI on state, customer AK 102 AK 103 IL 101 IL 105 OR 104 AK 201 IL 204 LA 202 NV 203 OR 205 AK 301 AK 304 IL 302 NV 303 TX 305 Data Partitioned Secondary Indexes are physically partitioned in the same ranges as the table

11 DPSI Limitations Depending on DB2 version, there are limits on whether a DPSI can contain duplicates Originally, DPSIs had to be unique Now they can be non-unique IF they contain the same key columns as the partitioning key DPSIs are great for partition concurrency Such as for utilities But they can be bad for SQL performance Unless the predicates limit scans to a single partition

12 Hash Access 12

13 Hash access to data Hashed row location New row Hash the key Place the row Existing row Existing row Existing row Existing row 13

14 Hash access to data Hashed row location Retrieved row Hash the key Get the row! Existing row Existing row New row Existing row Existing row 14

15 Hash access to data Access method ( H, HN and MH ) only for equals predicates Secondary indexes are also allowed For other predicates 15

16 Hash access to data warning Using a standard LOAD utility to load hash data will perform VERY badly Each record processed may need to be placed on a different page As input data is unlikely to be sorted into HASHED key sequence UNLESS using the IBM Load Utility with the IBM Utility Enhancement Tool Which can pre-sort input data into hashed key sequence prior to the load 16

17 Hash access to data Ideal for large tables with random access Up to 30% cpu reduction Good candidate tables include: Predominant access via a unique key No access by range scans Mostly accessed randomly Most queries satisfied by ONE row Size of table is relatively static Length of rows does not vary greatly At least 20 rows fit on a page 17

18 Index Include 18

19 Included columns in unique indexes Possible to INCLUDE additional columns in unique indexes Columns are NOT part of the unique constraint But, of course, ARE part of the key Can speed access to the data And possibly minimise indexes And perhaps lead to index-only CREATE UNIQUE INDEX ix1 ON tab1(col1,col2) INCLUDE (col3) ONLY allowed for unique indexes Not allowed for indexes on expressions 19

20 Included columns in unique indexes At first this seems to provide no real advantage After all aren t unique keys col1, col23, col3 and col1, col2, col3, col4 Both constrained by the uniqueness of cols 1, 2 and 3? Yes they are, BUT col1, col2, col3 INCLUDE col4 Only allows ONE row per unique combination of 1, 2 and 3 20

21 Included columns in unique indexes It is possible to INCLUDE new columns without changing the existing uniqueness constraint Also, existing indexes can have new columns added ALTER... ADD... INCLUDE COLUMN... May enable indexes to be combined An index for uniqueness and a similar index for data access 21

22 Index on Expression 22

23 SQL Compatibility Index on Expression In prior versions of DB2, you can only create an index on columns in a table From DB2 9, indexes may be created on expressions as well There are a number of (fairly obvious) limitations Data in the index IS stored as the expression result Queries referencing the same expression may take advantage of Index access

24 Index on Expression Example: CREATE INDEX PHIL.XEMP2 ON PHIL.EMP ( CONCAT(FIRSTNME,LASTNAME) ASC) USING STOGROUP SYSDEFLT ERASE NO FREEPAGE 0 PCTFREE 10 BUFFERPOOL BP0 CLOSE YES Would enable simple name searching SELECT * FROM EMP WHERE CONCAT(FIRSTNAME,LASTNAME) LIKE %GRAINGER%

25 Index on Expression When to use If particular expressions occur frequently in predicates, an index on that expression may be beneficial There will be an additional index maintenance cost But much faster data retrieval Can also provide access to data in new ways Case insensitive searches Soundex searches

26 Inline LOBs 26

27 Inline LOBs Only works with Universal Table Spaces In Reordered Row Format Part of the LOB will be stored alongside the relational data in the table space NOT in the LOB table space Existing LOBs can be ALTERed to be inline A small LOB (SLOB?) could be entirely inline 27

28 Inline LOBs Inline is slightly more expensive than varchar But cheaper than real LOB May also impact SQL not referencing LOB data Index on expression can be used on the in-line portion Remember, in-line LOB will reduce the space for relational data A maximum size in-line LOB will result in one row per page Good for LOB access Bad for repeated access to the same data page 28

29 INSTEAD OF Triggers 29

30 INSTEAD OF Triggers An INSTEAD OF trigger defines actions to be performed INSTEAD OF the triggering action Not BEFORE or AFTER They are created on VIEWS The UPDATE OF column_name clause is not allowed

31 INSTEAD OF Triggers With some limitations, they enable read-only views to be rendered updateable Must be ROW triggers No WHEN clause Not allowed for SELECT FROM INSERT/UPDATE/DELETE Not allowed for MERGE Provides a solution where views are used to isolate the data from the application For security or manageability

32 Materialised Query Tables 32

33 Materialised Query Tables A materialized Query Table (MQT) is a table that holds the result of a complex query against other tables If an SQL statement can use the MQT instead of performing the complex SQL (again) on the base table, performance gains will result Of course, there is a cost to maintaining the MQT 33

34 Materialised Query Tables 34

35 Materialised Query Tables MAINTAINED BY SYSTEM means that the data in the MQT can ONLY be refreshed by the REFRESH TABLE statement NOT by SQL or utilities NOTE This is NOT automatic MAINTAINED BY USER means that the data can be refreshed in any way The data is NEVER refreshed automatically or kept in synch with reality 35

36 Materialised Query Tables ENABLE QUERY OPTIMIZATION means that the DB2 optimiser can consider the MQT when determining access paths DISABLE QUERY OPTIMISATION means that Automatic Query Rewrite (AQR) will not take place CURRENT REFRESH AGE is also used to determine whether an MQT is recent enough to be a candidate for rewrite Only two options really ANY means any MQT is OK Anything else means NO MQT will be considered 36

37 Materialised Query Tables During the EXPLAIN process, DB2 will externalise to the PLAN_TABLE the name of the MQT rather than the table in the original query This will be in the TNAME column MQTs have their uses, but perhaps only in static environments Where the data is changing, it is difficult to keep the MQT reflecting the real data 37

38 Partition by Growth 38

39 Universal table spaces There are now TWO types of partitioning Partition by RANGE Key range partitioning that we ve always had Partition by GROWTH New partitions are added as data grows 39

40 UTS Partition by growth Starts off as a one partition table space As data is inserted/loaded the partition will fill up When it is full, another partition is automatically created No -904 No partitioning key You specify the limit to the number of partitions on the CREATE table space statement MAXPARTITIONS But PLEASE don t go crazy You can always alter MAXPARTITIONS later 40

41 UTS Partition by growth UTS PBG allows for much more data in a single table Do NOT get confused by the word partition Do NOT expect to be able to perform partition level operations Because rows are in partitions arbitrarily Not based on a key Some partition operations make no sense Rebalance LOAD INTO PART ALTER LIMITKEY Add/Rotate partitions Etc 41

42 Partition Rotation 42

43 More partition rotations DB2 Version 8 introduced ALTER... TABLE ROTATE FIRST TO LAST Very useful for applications that roll data through a partition key range Old data can be rolled away and a new partition created at the other end for new data Problems arise in the disconnection of logical and physical partition numbers Utilities (and SQL) work by PHYSICAL, on the other hand people tend to imagine LOGICAL Can lead to confusion and errors 43

44 More partition rotations DB2 10 gave us ALTER... TABLE ROTATE PARTITION nnn TO LAST Allowing ANY partition to be rotated to the end of the partition range The only partition you cannot rotate is the LAST one This effectively gives us DROP PARTITION Take the partition you don t want and rotate it to the end Yet more potential for confusion 44

45 More partition rotations Be VERY careful which partition number you specify on subsequent DDL or utility executions Remember PART is ALWAYS physical and NEVER logical Keep an eye on SYSTABLEPART PARTITION and LOGICAL_PARTITION columns Provided you have mechanisms in place to control the confusion, partition rotation is a useful addition BUT the potential for mistakes and irreversible damage is HUGE 45

46 Random Indexes 46

47 Random indexes As well as creating ASCending and DESCending indexes From DB2 9 we gained RANDOM indexes Any (or all) keys in an index can be RANDOM as well as ASC or DESC A RANDOM index is great for avoiding bottlenecks/hotspots BUT can only be used for equivalence predicates Not range predicates Has minimal use, as a random index is ONLY good for avoiding insert bottlenecks And uniqueness checking And single key data access No use for range predicates or result set ordering

48 Table clones 48

49 CLONEd tables Fast way to replace table contents Provides support for on-line LOAD REPLACE A CLONE inherits ALL of the attributes of the base table Including all indexes, constraints, triggers Clones are created with an ALTER statement 49

50 Cloned tables Once you have cloned a table, NOTHING can be done to change the structure of the base table Or the clone No on-line schema No rebalance etc. BUT if you create an index on the base table DB2 also creates the index on the clone And puts it in RBDP 50

51 Cloned tables The clone and base table can be exchanged DB2 switches the instance numbers in the catalog This is FAST Very like a switch phase 51

52 Cloned tables What was the clone is now the base table And vice versa So you can create a clone, manipulate the data offline and (almost) instantly put the data online And you can flip-flop as often as you like 52

53 Cloned tables Plans, packages etc are NOT invalidated Run RUNSTATS and REBIND if data has changed significantly During the exchange, both base and clone are drained Therefore not completely outage-free Duration of the exchange is a function of the NUMBER of objects, not their size The exchange MUST be committed before the data can be accessed VERY fast, and flexible, way of replacing the contents of a DB2 table Almost real-time 53

54 Temporal Data (again let s skip) 54

55 Truncate Statement 61

56 Truncate Very quickly delete all rows from a table Much faster than a DELETE.. FROM Even with segmented table spaces An IMMEDIATE option makes the TRUNCATE non-recoverable i.e. can t be rolled back 62

57 Truncate DROP/REUSE storage apply to multi table table spaces IGNORE delete triggers deletes the data WITHOUT firing delete triggers RESTRICT prevents truncation if delete triggers are present And the default is 63

58 Truncate Tables with Change Data Capture enabled will be truncated like a mass delete Tables with MLS defined, truncate will need to determine whether all rows can be deleted If a VALIDPROC is defined, truncate needs to verify validity of each row These will slow down the truncate So TRUNCATE is the fastest SQL way of emptying a table Especially if you have no triggers 64

59 XML Native Storage 65

60 DB2 for z/os support for XML Documents are not stored as strings So not comparable with any string datatype But are manipulated by various XML expressions and functions Including an XML predicate function There are some things you can t do with an XML column Sort Group Most predicates Primary, foreign or unique key Also, no host languages have XML data type manipulation support So XML data has to be manipulated as string data But this situation is changing all the time

61 XML indexes Using XPath notation, you can create indexes on your XML column CREATE UNIQUE INDEX XML_INDEX ON GRAPH02.XML_TABLE(XML_COLUMN) GENERATE KEY USING XMLPATTERN '/rss/channel/item/title' AS SQL VARCHAR(128) Yes, this IS a unique index And it DOES constrain the content of the node specified

62 XML update Three types of sub document update REPLACE (aka UPDATE) INSERT DELETE These changes may result in schema revalidation And may fail if schemas says change is invalid Only ONE updater per XML document Locking at base table (page/row) level Updaters of different nodes of the SAME document will contend 68

63 Native XML storage When does it make sense to store XML data natively? As opposed to just storing the document as a CLOB If you are merely using DB2 as a storage mechanism And only ever retrieve the whole document Then a CLOB is perhaps a better choice If you want to manipulate the document content And perhaps return subsets of it Or make changes to portions of the document Then native XML is the best choice 69

64 It Depends 70

65 Why does it always depend? IBM have continued to enhance DB2, often providing choices This means there is often no single Right Way of doing something The choice needs to take into account WHAT you are doing and WHY Only then can the right choice be made This tends to make these type of presentations a little vague Sorry 71

66 Advanced Design Considerations par Phil Grainger, BMC