DB2 for z/os Utilities Update

Transcription

1 Information Management for System z DB2 for z/os Utilities Update Haakon Roberts DE, DB2 for z/os & Tools Development haakon@us.ibm.com 1

2 Disclaimer Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. 2

3 Agenda Strategy CPU cost reduction REORG Load & Unload Backup & Recovery Statistics General Enhancements A View to the Future Summary 3

4 Strategy Support core function Reduce CPU, ET & resource consumption Maximize availability Remove constraints & limitations Simplify data management 4

5 The Increasing Importance of IBM DB2 Utilities DB2 Utilities Suite is required for core function enablement in DB2 DB2 Utilities Suite provides data & meta-data conversion capability REORG/LOAD row format conversion in DB2 9 REORG catalog/directory conversion during DB2 10 ENFM REORG non-disruptive meta-data changes in DB2 10 Pageset conversion, page size alteration, etc. REORG/LOAD inline LOBs in DB2 10 Including non-disruptive conversion of existing LOB data Utility support for hash pagesets in DB2 10 Including auto-estimation of required hash space in REORG Utility support for spatial indexes in DB2 10 Retrofitted to DB2 9 5 in addition to resizing pagesets, restoring clustering, reclaiming physical space etc.

6 Utilities & CPU Focus on total CPU elimination in addition to CPU reduction & ziip exploitation Real CPU cost reduction in V9 and more since DB2 utilities have been ziip-enabled since 2006 ziip offload for utility sort with DFSORT DB2 Sort product for further reduction in sort elapsed time & CPU consumption Flashcopy exploitation in DB2 10 dramatically reduces CPU consumption for COPY & reduces CPU for RECOVER & inline copies More ziip offload in DB2 10 with RUNSTATS More ziip offload for REORG in DB2 9 via APAR PM37622 Continued delivery of performance & CPU-reduction improvements via maintenance & new releases 6

7 DB2 Sort 1.3 Performance Benefits Use of DB2 Sort 1.3 with DB2 utilities, as compared with running DB2 utilities alone, may see: * Reduction of Sort CPU usage 7 Up to 74% reduction on machines with ziip engines Up to 43% reduction on machines without ziip engines Reduction of Utility CPU usage Up to 49% reduction on machines with ziip engines Up to 25% reduction on machines without ziip engines Reduction of Utility Elapsed Time * The information Up to 50% contained reduction this on slide machines is distributed with ziip AS IS. engines Performance data and results presented were determined in various controlled laboratory environments, using specific, limited test configurations, and are for reference Up purposes to 49% only. reduction Tests were on machines run against without the most ziip current engines versions of DB2 Sort and DB2 Utilities Suite generally available as of Aug. 17, Results reported for machines with ziip engines reflect a situation where all DB2 Sort program ziip eligible instructions are successfully dispatched to execute on available ziip processor(s). The results that may be obtained in other operating and production environments may vary significantly. Users of the product should verify the applicable results they might achieve for their specific environment. Aug. 17, 2012

8 DB2 10 REORG Avoidance & performance Reduced need for REORG INDEX List prefetch of index leaf pages based on non-leaf information for range scans Reduced need for REORG with compress on insert New REORGCLUSTERSENS RTS column If no clustering-sensitive queries then avoid REORG to restore clustering DSNACCOX and Automation Tool enhanced Improved performance for part-level REORG with NPIs & REORG INDEX Index list prefetch results in up to 60% elapsed time reduction 8

9 DB2 10 REORG Improved availability & removed restrictions - do things you could not do before Reduced application outage for REORG with inline stats Update catalog after dedrain REORG support for multiple part ranges LISTDEF PARTLEVEL(1:6,47,287:509) Retrofitted to DB2 9 Better availability with REORG SHRLEVEL CHANGE for all catalog/directory page sets Better restartability when repartitioning data with REORG SHRLEVEL REFERENCE CHANGE to remove REORP Better availability with REORG SHRLEVEL CHANGE for LOBs REORG SHRLEVEL NONE no longer supported in V10 NFM Will end rc0 but no REORG will be performed 9 FORCE option to allow REORG to cancel blocking threads

10 DB2 10 REORG New AUX keyword on REORG of partitioned base for improved LOB handling Permits rows to flow between partitions Allows REORG REBALANCE with LOB columns Allows ALTER of LIMITKEY with LOB columns Permits move of rows between parts on PBG REORG Permits deletion of corresponding LOBs on REORG DISCARD Default is AUX NO unless LOB objects required to complete REORG No XML column support for classic partitioned or PBR No mapping table change required Part 1 LOB Part 2 LOB 10

11 REORG APARs Retrofit of support of multiple part ranges More efficient, improved availability, exploit parallelism PK87762 & PM13259 (V9) E.g. REORG PART 1,45:71,500:503,4010 Note: LISTDEF parts will now process in a single REORG 2 implications to consider: May not have the disk space for sortwork or shadow pagesets OFFPOSLIMIT/INDREFLIMIT apply to entire set of partitions If cannot tolerate the above, then set new zparm to SERIAL PM25525 (V9) new PARALLEL keyword on REORG PM37293 (V9) new REORG_LIST_PROCESSING zparm to control REORG parallel processing when input is a part-level LISTDEF Online REORG materialisation of inline LOBs PM29037 (V10) 11 REORG to correct BRF/RRF mismatch after DSN1COPY PM40646 (V9)

12 REORG APARs Faster REORG with no NPIs PM37112 (V9) Improve SYSLGRNX processing reduced SYSLGRNX GETPAGEs by up to 97% REORG CPU reduction PM37630 (V9) Up to 10% CPU reduction through sort efficiency/avoidance More ziip offload for REORG PM37622 (V9) Up to 20% additional ziip offload REORG outage reduction Reduce final incremental processing in last log iteration, in one case reduced from 7 mins to zero PM46632 (V9) 12

13 REORG APARs Partition-level REORG performance PM55051 (V9) Cut elapsed time of REORG of 100 partitions by over 50% Multi-table REORG performance PM52469 (V9) 20 tables: 8% ET & CPU reduction Remove REORG scalability limit caused by mapping table index size limit 13 PM58177 (V9) Support mapping tables in PBGs Mapping index limit increased from 64Gb to 16Tb Part-level REORG NPSI insert performance improvement PM87403 (V9) 100m row table, 6 indexes LOAD RESUME 66% CPU reduction, 30% ET reduction REORG PART 9 45% CPU reduction, 26% ET reduction

14 DB2 10: LOAD & UNLOAD Remove MAX_UTIL_PARTS zparm Restriction removed for REORG in V9 LOAD of multiple compressed parts will copy a single dictionary to multiple parts Improved performance for LOAD REPLACE with LOB data Up to 50% elapsed time reduction Spanned record support for LOB/XML data LOBs & XML documents inlined in SYSREC with base data Option in addition to FRVs Seconds elapsed Performance & portability PDSE VBS LOBs unloaded 14

15 LOAD & UNLOAD APARs Faster handling of zero length LOBs in LOAD/UNLOAD PM12286 (V9) Faster UNLOAD TABLESPACE PM34858 (V9) More efficient scan of SYSTABSTATS Est. by one customer to reduce UNLOAD elapsed time from 44 mins to 55 secs Fast LOAD through index avoidance PM27962 (V9) New INDEXDEFER option to skip index key insert Leaves indexes or logical partitions in RBDP For LOAD RESUME or partition-level LOAD REPLACE with NPIs Can even skip unique indexes LOAD single part (5% of data) with 5 NPIs: Save 64% ET 15

16 LOAD & UNLOAD APARs LOAD/UNLOAD FORMAT INTERNAL PM19584 Unload and load data in true internal format 85% CPU & elapsed time reduction on UNLOAD 77% elapsed time, 56% CPU reduction on LOAD Supported by High Performance Unload LOAD PRESORTED PM19584 Avoid sort overhead when data already sorted in clustering order Up to 25% CPU reduction, 33% ET reduction depending on no. of indexes Works well with Utility Enhancement Tool PRESORT option 16

17 DB2 10: Backup & Recovery Dataset-level Flashcopy support COPY, RECOVER, REORG, LOAD, REBUILD INDEX, REORG INDEX New zparms & utility parms to govern Virtually eliminate CPU & elapsed time for large pagesets Create transaction-consistent image copies from COPY SHRLEVEL CHANGE Create partition-level inline image copies from REORG CPU time per object (z10) Elapsed time per object (z10) Seconds IC FC Seconds IC FC Object size (MB) Object size (MB)

18 DB2 10: Backup & Recovery VERIFYSET option to fail PIT recovery if entire set not included Base, LOB, XML & history objects ENFORCE NO option to avoid CHKP/ACHKP on PIT recovery of subset of set Improved performance due to avoidance of set checking (RI, aux) Fast recovery to point in time through new BACKOUT option Include indexes in RECOVER list to avoid the need to rebuild them Indexes must be COPY YES No imagecopy required though LOBs requires APAR PM45650 Image copy PIT recovery always with consistency since V9 Forward Backout 18

19 Backup & Recovery APARs LOB pageset support for RECOVER BACKOUT YES PM45650 (V10) Lost LOBs will be marked invalid if necessary Tablespace will be placed in AUXW Improved MODIFY RECOVERY support for SLBs PM24237 (V9) MODIFY will check for existence of SLB before setting copy-pending REPORT RECOVERY support for SLBs delivered in base V10 Faster image copy to tape PM23786 (V9) Improved tape mark handling when copying multiple pagesets to same tape One customer measured 40% elapsed time improvement SELECT from SYSLGRNX 19 PM35190 & PM42331 (V10) ISO(UR) enforced

20 DB2 10: Statistics KEYCARD deprecated and is now the default RUNSTATS PROFILE support for simplification Autonomic features through new stored procedures & catalog tables All catalog statistics columns made updatable RUNSTATS SHRLEVEL REFERENCE updates RTS TOTALROWS & TOTALENTRIES columns Up to 99% ziip-enablement for RUNSTATS Auto sampling rates & page sampling instead of row sampling Significant CPU & ET savings TABLESAMPLE SYSTEM AUTO DSNACCOX enhancements 20 Support hashed pagesets New RTS columns for SSD, other

21 DB2 10: Other CHECK DATA CHECK utilities will no longer set CHKP/ACHKP CHECK DATA enhanced for XML support Document validation Schema validation Automated exception table processing for XML documents Spatial index support for REORG, REBUILD INDEX & CHECK INDEX Removed UTSERIAL lock for greater utility concurrency LISTDEF support for DEFINED YES NO ALL Improved utility performance since unnecessary to build & then discard structures for undefined objects Default changed to DEFINED YES 21

22 Other utility APARs Retrofit utility spatial index support to V9 PM35200 (V9) Prevent read-only outage from CHECK SHRLEVEL CHANGE utility when Flashcopy support not available PM19034 (V9) New CHECK_FASTREPLICATION zparm Recommend setting to REQUIRED Empty lists generated by LISTDEF changed to return RC4 PM27099 (V10) Previously returned RC8 RC4 desirable, and necessary since default changed to DEFINED YES Improve DSN1COPY usability with REPAIR VERSIONS PM27940 (V9) For DSN1COPY of partitioned tablespaces 22 Allow utility restart when template uses &UNIQ or &UQ PM19063 (V9)

23 Subset of Utility Deliverables in the past 2 years REORG SHRLEVEL CHANGE for entire catalog & directory REORG SHRLEVEL CHANGE for LOB table spaces Additional RTS information for intelligent REORG Index leaf page list prefetch REORG INDEX performance due to index leaf page list prefetch REORG last log iteration performance Auto-correction of BRF/RRF mismatches ziip offload for RUNSTATS Dataset-level Flashcopy Point-in-time recovery by rollback Faster copy to tape REORG REBALANCE on partitioned tables with LOB columns ALTER LIMITKEY support for partitioned tables with LOB columns DB2 Sort LOAD/UNLOAD VBS support Faster UNLOAD TABLESPACE Ability to use SHRLEVEL REFERENCE or CHANGE to remove REORP state after alter of limitkey values REORG of multiple partition ranges Spatial index support Faster REORG with no NPIs ziip offload for REORG UNLOAD REORG sort performance LOAD/UNLOAD FORMAT INTERNAL LOAD PRESORTED LOAD INDEXDEFER Statistics profiles Page sampling & auto-sample rates for RUNSTATS DSNACCOX enhancements Removed UTSERIAL lock greater concurrency Improved utility restart with templates 23

24 A view to the future Even better availability Easier for REORG to break in on applications REORG SWITCH phase duration cut by up to 90% Online rebalance of data across partitions Up to 50% faster offline load processing Even better performance & reduced resource consumption Reduce/eliminate need for RUNSTATS through inline stats improvements Over 80% online load elapsed time improvement More utility parallelism & greater parallelism control More ziip exploitation, including up to 100% for load when clearing parts Advanced statistics recommendations from DB2 optimizer 24

25 A view to the future Simplified data management & improved usability Automated REORG mapping table management Intelligent default settings for REORG Better PBG management through deletion of unused partitions Improved dataset management & faster recovery with part-level inline image copies Improved XML handling with Crossloader support System cloning improvements Lift many point in time recovery restrictions More information & greater transparency through DISPLAY UTILITY improvements Simplified statistics gathering through profile enhancements 25

26 Summary Unparalleled investment in utilities & tools IBM Utilities Suite is essential for exploitation of major DB2 enhancements Support of core DB2 function from day 1 of GA Expect continued delivery of enhancements on release boundary, and in maintenance stream when prudent Continued focus on: Elimination of application impact from utilities Elapsed time & CPU consumption reduction Resource consumption reduction Reduction in complexity & automation improvements Synergy with IM Tools 26