PBR RPN & Other Availability Improvements in Db2 12 Haakon Roberts IBM Session code: A11 07.11.2018 11:00-12:00 Platform: Db2 for z/os 1
Disclaimer IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Agenda Revisiting 10 byte log RBAs V12 Scalability PBR RPN Online schema Recent Developments Future Ideas Summary
10 byte log RBA revisited - background Prior to V11, RBAs and LRSNs were 6 bytes V11 supports 10 byte RBAs and LRSNs LRSNs due to run out in 2042 if no delta RBA: D34C4322BAEC converts to 00000000D34C4322BAEC LRSN D34C4322BAEC converts to 00D34C4322BAEC000000 1 yottabyte Approx 30,000 more years for LRSN
10 byte log RBA revisited - background BSDSs converted using DSNJCNVT Must be converted prior to migration to V12 Pagesets may be converted to extended format CREATE REORG LOAD REPLACE REBUILD INDEX
10 byte log RBA revisited - background ZPARMs OBJECT_CREATE_FORMAT BASIC/EXTENDED Default is BASIC UTILITY_OBJECT_CONVERSION BASIC/EXTENDED/NONE/NOBASIC Default is NONE (no conversion)
10 byte log RBA revisited - thresholds F00000000000 DSNJ032I issued every log switch or restart Until BSDSs are converted FF8000000000 Soft update limit Updates to basic format user pagesets fail -904 00C2026D FFFF00000000 DSNJ033E Db2 shut down and only restartable in maint mode until BSDSs converted Or RBA is reset FFFFFFFFFFFF Hard update limit Updates to all basic format pagesets fail -904 00C2026E
Beware Customers convert the BSDSs but not any pagesets No longer get DSNJ032I messages No other messages issued until hit the FF8000000000 limit causing updates to fail and application outages Then convert user pagesets, since no errors against catalog/directory This accelerates logging until hit FFFFFFFFFFFF limit at which point no more conversions are possible because conversions need to update cat/dir but cat/dir is in basic format Customers hitting limits convert to datasharing thinking that that will avoid the need to convert pagesets to extended format If BSDS not converted, and RBA is FF8000000000 then LRSN will have delta applied to make LRSN FF8000000000 This will give a max of 4 years 5 months before 6 byte LRSN runs out If LRSN is higher, e.g. FFFF00000000, then this will give only 19 hours before 6 byte LRSN runs out Converting BSDS at this stage will not help FFFF00000000 converts to 00FFFF00000000000000 Once we move to 01000000000000000000 then the pageset must be extended format to support this value
Recommendations Convert BSDSs and pagesets once in 11 NFM No benefit to basic format Converted BSDSs can save CPU Converted pagesets can avoid LRSN spin in data sharing Do not assume everything is fine just because you converted the BSDSs and no longer get DSNJ032I If you hit the FF0000000000 (or 00000000FF0000000000) limit, do not ignore the catalog and directory Cat/dir pagesets MUST be converted before FFFFFFFFFFFF limit is hit What is Db2 doing to help? All documentation updated to highlight the importance of conversion and in particular cat/dir conversion New APAR will remove zparms so new objects will be extended format and pagesets will be converted to extended format by utilities New APAR will issue message to console once per life of a Db2 system when a basic format pageset is opened Note: In datasharing there is no need to convert all pagesets before going beyond RBA 00000000FFFFFFFFFFFF
Scalability Improved storage management for EDM pools No longer fail if needed storage exceeds limit EDMSTMTC max EDM statement cache pool EDMDBDC Max DBD pool EDM_SKELETON_POOL Max skeleton pool Automatic management of LOB and XML pools Removed LOBVALA/LOBVALS/XMLVALA/XMLVALS Settings are 2Gb per user and 50Gb per subsystem Larger active log datasets Max size increased from 4Gb to 768Gb No change in max number of active log datasets 10
Partition Management prior to V12 DB2 page numbers are absolute 5 byte RIDs describe partition, page and row within page DSSIZE limits are dependent upon number of partitions, creating complex scalability constraints Alter of DSSIZE requires REORG of entire table space Running out of space in a partition causes an application outage and resizing may not be possible without dropping and recreating the table space
Partition Management - Relative Page Numbering Simplicity, usability, scalability, availability Support relative page numbers for range partitioning CREATE or ALTER TABLESPACE PAGENUM(RELATIVE) Alter from either classic partitioned or PBR No support for tablespaces with limitkeys truncated at 40 bytes Must be fixed before converting to PBR RPN Deferred alter followed by materializing REORG Table space - level REORG required for conversion No PIT recovery to prior to materializing REORG Separate page numbers from partition numbers
Partition Management - Relative Page Numbering Support partition-level DSSIZE up to 1Tb per part Regardless of number of partitions or page size Max table space size increase from 16Tb to 4Pb Online alter to increase DSSIZE - immediate Online alter to decrease DSSIZE - pending alter requiring table space - level REORG RID increase from 5 bytes to 7 bytes requiring mapping table change New mapping table format optional until new function enabled REORG requires part-level inline image copies
Sample CREATE TABLESPACE CREATE TABLESPACE TSPBR IN DB1 PAGENUM RELATIVE DSSIZE 8G NUMPARTS 3 (PARTITION 1 DSSIZE 3G, PARTITION 3 DSSIZE 9G); SYSTABLEPART PARTITION PAGENUM DSSIZE 1 R 3,145,728K 2 R 8,388,608K 3 R 9,437,184K 14
Sample ALTER TABLESPACE ALTER TABLESPACE TSPBR IN DB1 ALTER PARTITION 1 DSSIZE 5G ALTER PARTITION 3 DSSIZE 12G; SYSINDEXPART PARTITION DSSIZE 1 5,242,880K 2 8,388,608K 3 12,582,912K 15
Sample CREATE INDEX CREATE INDEX IXPI ON TB1(COL1) PARTITIONED DSSIZE 1G; SYSINDEXPART PARTITION DSSIZE 1 1,048,576K 2 1,048,576K 3 1,048,576K 16
Sample ALTER INDEX ALTER INDEX IXPI ALTER PARTITION 1 DSSIZE 3G ALTER PARTITION 3 DSSIZE 8G; SYSINDEXPART PARTITION DSSIZE 1 3,145,728K 2 1,048,576K 3 8,388,608K 17
Partition Management - Relative Page Numbering New format log records support relative page numbering for all page sets Not just range partitioned From before activation of new function onwards, so V11 products need toleration code PI34223 delivers DB2 11 support for DB2 12 log records Improved DSN1LOGP serviceability New PART option, filtering & enhanced log record formatting
Insert Partition ALTER TABLE ADD PARTITION ENDING AT xxx Alternative syntax ALTER TABLE ADD PARTITION ENDING AT xxx ALTER PARTITION n ENDING AT yyy Physical partition number is at the end Logical partition number based on position Attributes inherited from SYSTABLESPACE values Or previous logical partition if SYSTABLESPACE values are null Pending change, affected partition placed in AREOR Only affected partition(s) need to be reorged PBR only No PIT recovery prior to materializing REORG
Online Schema ALTER INDEX COMPRESS YES places index in AREOR instead of RBDP UTS table spaces only Materialized through REORG INDEX or REORG TABLESPACE Support pending column-level ALTERs Avoid availability constraints & conflict with other deferred alters ALTER TABLE ALTER COLUMN SET DATA TYPE Materialize changes through online table space level REORG Governed by DDL_MATERIALIZATION zparm ALWAYS_IMMEDIATE ALWAYS_PENDING UTS only TRANSFER OWNERSHIP Database objects DATABASE, INDEX, STOGROUP, TABLESPACE, TABLE, VIEW
REORG Fail REORG on Flashcopy failure if wholly reliant on inline FlashCopy Previously left object in copy-pending, now leave object in RW No need for TERM UTIL Enable full exploitation of Flashcopy in REORG Prevent copy-pending on LOB tablespace during REORG of PBG Growth of new PBG partition during REORG LOG phase would leave new LOB tablespace in copy-pending
Recovery Support recovery to PIT prior to materializing REORG for PBG table spaces Equivalent to PBR support delivered in V11 RECOVER now defaults to PARALLEL Up to 35% ET improvement New RECOVER SCOPE UPDATED option for PIT recovery Skip recovery for non-updated pagesets Retry auto-grecp and auto-lpl recovery Periodically up to 3 times based on internal timer For indoubt or PA URs, LPL recovery attempts commence once indoubt or postponed abort is resolved
General Better utilities concurrency COPYTOCOPY, MODIFY RECOVERY, MERGECOPY now compatible with other utilities IFI306 records returned in correct version Avoid need to REORG prior to enabling subscriptions after adding or altering columns Auto-bind releases resources prior to application commit Retrofit to V11 in PI46379 Single phase migration No ENFM phase New function activated through ACTIVATE FUNCTION LEVEL command APPLCOMPAT rules, fallback rules continue to apply
Recent Availability Enhancements Prevent RTS externalization from interfering with CATMAINT during migration PI49628 (V11) Mark utility I/O lock as critical Prevent DB2 failures when short on IRLM storage PI81019 (V11) Allow update of partitioning key columns For tables created prior to V5 APAR PQ16946 These columns are flagged as non-updatable Convert them to be updatable during conversion to table-controlled partitioning or conversion to PBR or PBR RPN PI73421 Allow externalization of RTS even if tablespace is in UTRW state PI87403 24
Recent Availability Enhancements Improved support for ALTER ADD PARTITION in V11 Prior to PI42999 If limitkey on last partition is MAXVALUE, then it must be lowered before adding new partition. In 11, alter of limitkey is a deferred alter, placing partition in AREOR state Attempt to ALTER TABLE ADD PARTITION fails since it is an immediate alter which is incompatible with deferred alter This forces a REORG of the last partition which can discard data records that need to be preserved then loaded back into table A later REORG is required after the new partition is added With PI42999 ALTER TABLE ALTER PARTITION limitkey is changed to be an immediate change if altering the last partition and the high key is MAXVALUE (or MINVALUE) The partition is left in REORP, allowing for ALTER TABLE ADD PARTITION and a single REORG to resolve Storage reduction for UNLOAD of XML data >4Gb per utility down to 10s of Kb PI84464 25
Recent Availability Enhancements LOAD REPLACE SHRLEVEL REFERENCE LOAD utility pre-validation Including CHECK constraints Support usual drain options, including SWITCHTIME Works for partition load also Logical NPSI partitions left in RBDP Like INDEXDEFER PI67793 & PI87474 V12 APAR: PI69095 Shadow Input data Target
Recent Availability Enhancements Allow ROTATE PARTITION on MQTs or tables with MQTs defined PH00194 Undo a restriction that was added by PI77487 Need to invalidate static packages for both ROTATE and alter of limitkeys on MQTs Avoid package invalidation on PBR conversion PH01819 (V12 only) 27
Recent Availability Enhancements Improved DDL break in on dynamic SQL Serial nature of invalidation process in dynamic statement cache makes it difficult for DDL to break in PH00637 (V12 only) Before stmts / uow ddl total ddl fail sql total sql fail 2 23 19 2522 407 10 6 17 14 2360 461 10 time (min) After stmts / uow ddl total ddl fail sql total sql fail 2 30 0 2440 600 10 6 30 0 1720 600 10 time (min) 28
Future ideas Improved online schema processing More online schema changes Improved support for REORG part-level inline image copies to tape
Summary Continuing focus on improved availability & elimination of application impact More deferred ALTERs and materializing REORGs More granular processing and increased usability and flexibility in schema Improved and less disruptive autobind processing Less disruptive migrations and continuous delivery
Haakon Roberts IBM haakon@us.ibm.com Session code: A11 Please fill out your session evaluation before leaving! 31