SQLADRIA SEMINAR. Ljubljana, 13 April Sponsored by

Transcription

1 SQLADRIA SEMINAR Ljubljana, 13 April 2012 Sponsored by

2

3 Relational Database Users Committee for Croatia and Slovenia Odbor korisnika relacijskih baza podataka Hrvatske i Slovenije Odbor uporabnikov relacijskih baz podatkov Hrvaške in Slovenije SQLAdria Seminar 10:00 Opening and presenting sponsor :00 V10 Migration Planning and Experiences? Gain insight into V9 and V10 Performance and save costs? John Campbell IBM 12:00 Coffee Break 12:15 14:00 Advanced Backup and Recovery using Flashcopy? V10 Virtual and Real Storage Management? John Campbell IBM 14:00 Lunch 15:00 15:30 User theme: Loading a Data Warehouse Zdenka Gazvoda, Jure Lovšin Zavod za zdravstveno zavarovanje Slovenije Sponsored by: Ljubljana, 13 th April 2012 Hotel Slon CTK Rijeka, Školjić 5d, Rijeka, tel , , fax , sqladria@ctk-rijeka.hr

4

5 Participants of the SQLAdria Seminar Ljubljana, 13 th April 2012 No Name Surname Company E mail 1 Erika Polanič Informatika d.d. erika.polanic@informatika.si 2 Andrej Furman Informatika d.d. andrej.furman@informatika.si 3 Barbara Rečnik Informatika d.d. barbara.recnik@informatika.si 4 Mojca Kosirnik Informatika d.d. mojca.kosirnik@informatika.si 5 Marjana Glavan Informatika d.d. marjana.glavan@informatika.si 6 Karmen Kristan Kamenečki Informatika d.d. karmen.kristan kamenecki@informatika.si 7 Valerija Koletnik Informatika d.d. valerija.koletnik@informatika.si 8 Brane Bruvo Informatika d.d. brane.bruvo@informatika.si 9 Matej Jozelj Informatika d.d. matej.jozelj@informatika.si 10 Aleš Vicozi Informatika d.d. ales.vicozi@informatika.si 11 Mateja Jankovič Informatika d.d. mateja.jankovic@informatika.si 12 Ana Ramljak Zagrebačka banka d.d. ana.ramljak@unicreditgroup.zaba.hr 13 Nenad Vidak Zagrebačka banka d.d. nenad.vidak@unicreditgroup.zaba.hr 14 Sven Lovrenčić Zagrebačka banka d.d. sven.lovrencic@unicreditgroup.zaba.hr 15 Mirna Kos Zagrebačka banka d.d. mirna.kos@unicreditgroup.zaba.hr 16 Rok Žigon Elektro Ljubljana rok.zigon@elektro ljubljana.si 17 Marko Žagar Elektro Ljubljana marko.zagar@elektro ljubljana.si 18 Viktor Ulčar Elektro Ljubljana viktor.ulcar@elektro ljubljana.si 19 Vera Trtnik Elektro Ljubljana vera.trtnik@elektro ljubljana.si 20 Mija Glogoški Elektro Ljubljana mija.glogoski@elektro ljubljana.si 21 Luka Pipp NLB d.d. luka.pipp@nlb.si 22 Majda Zajc Markelj NLB d.d. majda.zajcmarkelj@nlb.si 23 Mateja Pečarič Žunič NLB d.d. mateja.pecaric@nlb.si 24 Peter Konda NLB d.d. peter.konda@nlb.si 25 Branko Božič NLB d.d. branko.bozic@nlb.si 26 Istok Sajevic NLB d.d. istok.sajevic@nlb.si 27 Mitja Makovec NLB d.d. mitja.makovec@nlb.si 28 Goran Bavčar NLB d.d. goran.bavcar@nlb.si 29 Marko Koprivnikar IBM Slovenija d.o.o. marko.koprivnikar@gmail.com 30 Iva Mesarić HZMO iva.mesaric@mirovinsko.hr 31 Željko Blagajić HZMO željko.blagajic@mirovinsko.hr 32 Goran Petrinjac HZMO goran.petrinjac@mirovinsko.hr 33 Dragica Šmintić Apis IT d.o.o. dragica.smintic@apis it.hr 34 Rok Brnot ZPIZ rok.brnot@zpiz.si 35 Rok Medja ZPIZ rok.medja@zpiz.si 36 Viljano Pavletić ZPIZ viljano.pavletic@zpiz.si 37 Darija Abramič ZPIZ darija.abramic@zpiz.si 38 Mirjana Kregar ZZZS mirjana.kregar@zzzs.si 39 Evgen Gržanič ZZZS evgen.grzanic@zzzs.si 40 Zdenka Gazvoda ZZZS zdenka.gazvoda@zzzs.si 41 Janez Miko ZZZS janez.miko@zzzs.si 42 Tine Košir ZZZS tine.kosir@zzzs.si 43 Tatjana Šumec ZZZS tatjana.sumec@zzzs.si 44 Andrej Koporc ZZZS andrej.koporc@zzzs.si 45 Niko Schlamberger Slovensko društvo informatike niko.schlamberger@gmail.com 46 Olivera Stanić CTK Rijeka, SQLAdria ostanic@ctk rijeka.hr

6

7 SQLADRIA SEMINAR Ljubljana, 13 April 2012 V10 Migration Planning and Experiences? Gain insight into V9 and V10 Performance and save costs? John Campbell IBM Sponsored by

8

9 IBM DB2 10 Migration Planning and Very Early Experiences Part 1 John Campbell Distinguished i Engineer DB2 for z/os Development Notes: John Campbell1/UK/IBM@IBMGB CampbelJ@uk.ibm.com 1 Agenda Keys to customer migration success Performance and Scalability BIND, REBIND and EXPLAIN Availability Migration Planning Removal of DDF Private Protocol Security Considerations when removing DDF Private Protocol Other Summary 3 Objectives Share lessons learned, surprises, pitfalls Provide hints and tips Address some myths Provide additional planning information Provide usage guidelines and positioning on new enhancements 2 Keys to customer migration success 1. Plan for continual application of preventative service Need to stay more current on HIPERs at this stage in the release take up cycle Apply preventative service every 3 months Two major and two minor releases Refresh of the base every 6 months ( major ) Each base should be based on latest quarterly RSU as opposed use of PUT In addition, two minor packages covering HIPERs and PEs in between time Augment by exploiting Enhanced HOLDDATA on a weekly basis before production cutover and continue thereafter Identify and pull all applicable HIPERs and PE fixes Expedite the most critical PTFs into production 2. Perform application regression and stress testing is the best way to keep fires away from production 3. Build a realistic project plan Avoid crash project Allow contingency for bumps in the road Involve applications teams early Investigation of incompatible changes and fix up Testing 4

10 CST and RSU example CST4Q10 RSU1012 RSU1101 RSU1102 CST1Q11 RSU1103 All service through end Sept 2010 not already marked RSU All service through end Dec 2010 not already marked RSU + H&PE through end Nov 2010 H&PE through end Dec 2010 H&PE through end Jan H&PE through end Feb 2011 Available at the beginning of January 2011 Available at the beginning of February 2011 Available at the beginning of March 2011 Available at the beginning of April 2011 Base: Sep 2010 Base: Sep 2010 Base: Sep 2010 Base: Dec 2010 H&PE: Nov 2010 H&PE: Dec 2010 H&PE: Jan 2011 H&PE: Feb 2011 H&PE = HIPER/Security/Integrity/Pervasive PTFs + PE resolution (and associated requisites and supersedes) 5 Performance and Scalability Assumes no major access path regressions On Day 1 in production in CM without any changes (e.g., no rebind, no use of 1MB page size) there may be customers who see zero % improvement and even some will see degradation Why? SPROCs disabled, puffing of run time structures for migrated packages from V8 or V9, etc To maximise the performance improvements must: REBIND static SQL packages Use PGFIX=YES bufferpools with sufficient 1MB real storage page frames to 100% fully back the requirement from PGFIX=YES bufferpools Seeing 0-10% improvement after REBIND and use of 1MB real storage frames Need to look at total CPU resource consumption picture across Acctg Class 2 TCB Time (Accounting Trace) DB2 System Address spaces (Statistics Trace) 7 Performance and Scalability Many opportunities for price/performance (cost) improvements Major theme of this release Most welcome to our customers Customers intimidated by the marketing noise about improved performance Expectation of their CIO For some of their workloads not seeing improvements in CPU and elapsed time Conversely see big improvements for certain workloads Small workloads can skew expectations on savings Some measurements and quotes are insanely positive Should be ignored How to extrapolate and estimate for production mixed workload? Estimation with accuracy and high confidence not practical Benchmarking effort would be required Very important to correctly level set customer performance expectations Customers should not spend any performance benefits until they see them 6 Performance and Scalability Customers should expect to see some increase in real storage consumption (10-30%) Must also factor in MAXSPACE requirement for DB2 dumps (approx 16GB) Avoid very long dump capture times and bad system performance Critical for V10 serviceability 8

11 Performance and Scalability The 0-10% CPU reduction is based on the DB2 portion of a given application workload Customer value driven on how sub-capacity workload licensing works Based on 4-hour rolling average MSU utilisation Highest rolling average figure for each month used to calculate software charges for all MLC products (IBM and non-ibm) Provided DB2 forms a significant component of the total MSU usage during peak period, any MSU savings will translate directly to MLC savings Typically this is the online day - mid morning and mid afternoon So for example - this may be driven by CICS-DB2 workload where the DB2 portion of the workload only represents 40-60% of the total path length So the 0-10% may represent only 0 to 6% (i.e., needs to be discounted) Investigate how much CPU is used in the 4-hour period for DB2 work (SQL) Evaluate V10 price bands under WLC pricing vs. V10 MSU savings Factor in the impact on overall z/os software stack cost reduction z/os, CICS, MQ 9 Performance and Scalability Opportunities for additional price/performance improvements driven by DBM1 31-bit VSCR supported by additional real storage include More use of persistent threads with selective use of RELEASE(DEALLOCATE) High Performance DBATs CICS Protected ENTRY Threads CICS Unprotected ENTRY Threads with queuing Typical savings 0-10%, may be more Increasing MAXKEEPD to improve Local Dynamic Statement Cache hit ratio and reduce the number of short prepares Sysplex/Data sharing Group consolidation So for example, 8-way to 4-way Reduced cost of data sharing Very important to correctly level set customer performance expectations Customers should not spend any performance benefits until they see them 11 Performance and Scalability Sub capacity pricing Chart courtesy of Cristian Molaro, taken from White Paper: Getting the financial benefits of DB2 10 for z/os 10 Performance and Scalability Workload Customer Results CICS online transactions Approx. 7% CPU reduction in DB2 10 CM after REBIND, additional reduction when 1MB page frames are used for selective buffer pools CICS online transactions Approx 10% CPU reduction from DB2 9 CICS online transactions Approx 5% CPU reduction from DB2 V8 CICS online transactions 10+% CPU increase Distributed Concurrent Insert 50% DB2 elapsed time reduction, 15% chargeable CPU reduction after enabling high performance DBAT Data sharing heavy concurrent insert 38% CPU reduction Queries Average CPU reduction 28% from V8 to DB2 10 NFM Batch Overall 20-25% CPU reduction after rebind packages 12 12

12 Performance and Scalability Workload Customer Results Multi row insert (data sharing) 33% CPU reduction from V9, 4x improvement from V8 due to LRSN spin reduction Parallel Index Update 30-40% Elapsed time improvement with class 2 CPU time reduction Inline LOB SELECT LOB shows 80% CPU reduction Include Index 17% CPU reduction in insert after using INCLUDE INDEX 20-30% CPU reduction in random access Hash Access 16% CPU reduction comparing Hash Access and Index-data access. 5% CPU reduction comparing Hash against Index only access Further improvements delivered late in the beta program Performance and Scalability Query performance enhancements No REBIND required for Index list prefetch INSERT index read I/O parallelism Workfile spanned records SQLPL performance High performance DBATs Inline LOBs 15 Performance and Scalability Measurements of IBM Relational Warehouse Workload (IRWW) with data sharing Base: DB2 9 NFM REBIND with PLANMGMT EXTENDED DB2 9 NFM DB2 10 CM without REBIND showed 1.3% CPU reduction DB2 10 CM REBIND with same access path showed 4.8% CPU reduction DB2 10 NFM brought 5.1% CPU reduction DB2 10 CM or NFM with RELEASE DEALLOCATE 12.6% CPU reduction from DB CM CM REBIND NFM DEALLOC 14 Performance and Scalability Query performance enhancements REBIND required for Use of RELEASE(DEALLOCATE) Early evaluation of residual predicates IN-list improvements (new access method) SQL pagination (new access method) Query parallelism improvements Index include columns More aggressive view/table expression merge Predicate evaluation enhancements RID list overflow improvements Execute RUNSTATS before REBIND When coming from V8, to collect improved index statistics including CLUSTERRATIOF When coming from V9, if do not already include the KEYCARD option of RUNSTATS 16

13 Performance and Scalability Potential for access path regression when using OPTIMIZE FOR 1 ROW Used by customers as a hint to discourage use of sort or list prefetch Sometimes applied as an installation SQL coding standard DB2 access path selection has always been cost based V10 hammer change Excludes the sort access plan candidates Remaining sort avoidance access plans compete on cost lowest cost wins If no sort avoidance access plans, then sort access plans remain and compete on cost Seeing increasing evidence of access path regression when multiple candidate indexes available e.g., DB2 using alternate index with lower MATCHCOLS value because there is no sort Solutions Change application to code OPTIMIZE FOR 2 ROWS Alter an existing index or create a new index that would support both sort avoidance and index matching (if predicates allow) APAR PM56845 now open to provide option for OPTIMIZE FOR 1 ROW to allow sort access plans 17 Performance and Scalability Bufferpool Page Classification Sequential, dynamic, and list prefetch are all treated the same Pages are marked sequential at the time they are prefetched In V8 These pages were reclassified as random when subsequently touched via getpage In V9 and V10 None of them will be re-classified to random on a getpage CF utilisation issue Root cause is the way GBP data in the CF is deleted in V10 CF DELETE call option being invoked by V10 deletes both data and directory entries in one CF call But the code path used with name class mask requires more internal CFCC serialisation and can take longer Results in significantly more re-drives of the DELETE requests observed with V10 Increased volume of re-drives causes the significant increase in CF utilisation See APAR PM51467 for this issue 19 Performance and Scalability Increase in DB2 system address space CPU resource consumption DBM1 SRB More use of prefetch Row level sequential detection and progressive prefetch INSERT index read I/O parallelism Index list prefetch when disorganised index After BIND, more use of list prefetch ziip offload for prefetch and deferred write Seeing 50-70% ziip offload achieved DBM1 TCB Closing of high use CLOSE=YES datasets when hitting DSMAX because of stale list See APAR PM56725 for this issue MSTR TCB Increase related to real storage monitoring which was introduced (APAR PM24723) DB2 is calling a z/os RSM service for COUNTPAGES function which serialised the frame access with spin loop CPU increase especially when multiple DB2 subsystems running on the same LPAR See z/os APAR OA37821 and corresponding DB2 APAR PM49816 for this issue 18 Performance and Scalability DB2 10 and z196 synergy Taking the general case, performance improvement from V9 to V10 observed on z10 processor should be in same range on z196 processor as long as they are measured on the same number of processors Expectation is still in the 5-10% range Apart from MIPs improvement, z196 provides Higher cache hit ratio thus better scalability as number of processors per LPAR increases (more than 16 processors per LPAR) V10 performance on z196 Scales better with more processors per LPAR than z10 Can run with higher number of concurrent threads IBM measurement shows 20% ITR improvement from V9 (with a few benchmark specials) compared to V10 on z way with IRWW-like workload Measurement is extreme case Will only apply to very high end customers Not a general message Why does V10 run better on z196 Latch contention reductions, 1MB real storage page frame size, general path length 20

14 Performance and Scalability Use of 1MB size real storage page frames on z10 and z196 Long term bufferpool page fix was introduced in V8 to reduce CPU Many customers reluctant to use PGFIX=YES Potential for real storage shortage because running too close to the edge of the amount of real storage provisioned Customers do understand the value, but it only applies for a few hours each day But page fix is a long term decision In most cases requires DB2 recycle to change attribute 75% cost reduction on real storage on z196 (USD1.5K vs. USD6K) 21 Performance and Scalability Use of 1MB size real storage page frames on z10 and z196 Useful commands DB2 -DISPLAY BUFFERPOOL(BP1) SERVICE=4 Useful command to find out how many 1MB size page frames are being used Especially useful when running multiple DB2 subsystems on the same LPAR See DSNB999I message MVS -DISPLAY VIRTSTOR,LFAREA Show total LFAREA, allocation split across 4KB and 1MB size frames, what is available See IAR019I message 23 Performance and Scalability Use of 1MB size real storage page frames on z10 and z196 Potential for reduced for CPU through less TLB misses CPU reduction based on customer experience 0 to 6% Buffer pools must be defined as PGFIX=YES to use 1MB size page frames Must have sufficient total real storage to fully back the total DB2 requirement Involves partitioning real storage into 4KB and 1MB size page frames Specified by LFAREA xx% in IEASYSnn parmlib member and only changeable by IPL 1MB size page frames are non-pageable If 1MB size page frames are overcommitted, DB2 will use 4KB size page frames Recommendation to add 5-10% to the size to allow for some growth and tuning Must have both enough 4KB and enough 1MB size page frames Do not use 1MB size real storage frames until running smoothly on V10 Make sure any critical z/os maintenance is applied before using 1MB size real storage page frames 22 Performance and Scalability Exceptions where CPU regression for very light OLTP transactions Skinny packages with few simple SQL Package allocation cost overrides benefit from SQL optimizations in V10 APAR PM31614 may solve this by improving package allocation performance Good candidate for the use of persistent threads with RELEASE(DEALLOCATE) and will help compensate 24

15 Performance and Scalability DBM1 31-bit Virtual Storage Constraint Relief with 64-bit SQL run time V7 V8 V9 V10 16 ExaByte=2 64 DBM1 address space 2 GigaByte=2 31 IRLM locks DDF ctrl-blks Castout bfrs Compression DBD Cache Global DSC Ridpool Sortpool SK-CT/PT + Castout bfrs Compression DBD Cache Global DSC Ridpool Sortpool CT/PT Thread + Castout bfrs Real storage Compression DBD Cache Global DSC Ridpool SK-CT/PT Sortpool All EDMPool Thread DSMAX CT/PT Thread DSMAX Pointer DSMAX Dataspaces CTHREAD+MAXDBAT=2000 CTHREAD+MAXDBAT=20000 practical limit ~a few hundreds ~a few thousands 25 Performance and Scalability DBM1 31-bit Thread Storage V9 vs. V10 Initially but corrected prior to GA V9 V10 CM V10 CM Rebind 27 Performance and Scalability DBM1 31-bit Virtual Storage Constraint Relief with 64-bit SQL run time Available in CM Requirement to REBIND static SQL packages to accrue maximum benefit Very good results achieved (up to 90% VSCR) Have high degree of confidence that problem addressed Real world proposition: 500 -> threads plus Limiting factors now on vertical scalability (# number of threads, thread storage footprint) Amount of real storage provisioned on the LPAR Log latch (LC19) contention ESQA/ECSA (31-bit) storage 26 Performance and Scalability DBM1 31-bit Thread Storage V9 vs. V10 as at GA after Fix V9 V10 CM V10 CM with Fix 28

16 Performance and Scalability DBM1 31-bit Virtual Storage Constraint Relief with 64-bit SQL run time Major customer opportunities here for 31-bit VSCR and improved price/performance Potential to reduce legacy OLTP transaction CPU cost through use of More CICS protected ENTRY (persistent) threads More use of RELEASE(DEALLOCATE) with next/existing persistent threads Potential to reduce CPU for DRDA transactions by using High Performance DBAT Must be using CMTSTAT=INACTIVE so that threads can be pooled and reused Packages must be bound with RELEASE(DEALLOCATE) to get reuse for same connection MODIFY DDF PKGREL(BNDOPT) must also be in effect Do not to overuse RELEASE(DEALLOCATE) on packages > Will drive up the MAXDBAT requirement Potential to reduce CPU when using KEEPDYNAMIC(YES) e.g., SAP Increase MAXKEEPD to improve Local Dynamic Cache Hit Ratio and reduce the number of short prepares Must provision additional real storage to back the requirement for each opportunity 29 Performance and Scalability DBM1 31-bit Virtual Storage Constraint Relief with 64-bit SQL run time High Performance DBATs (Hi-Perf DBATs) is a new type of distributed thread Must be using CMTSTAT=INACTIVE so that threads can be pooled and reused Packages must be bound with RELEASE(DEALLOCATE) to get reuse for same connection and -MODIFY DDF PKGREL(BNDOPT) must also be in effect When a DBAT can be pooled after end of client's UOW Now DBAT and client connection will remain active together > Still cut an accounting record and end the enclave After the Hi-Perf DBAT has been reused 200 times > DBAT will be purged and client connection will then go inactive All the interactions with the client will still be the same in that if the client is part of a sysplex workload balancing setup, it will still receive indications that the connection can be multiplexed amongst many client connections IDTHTOIN will not apply if the if the Hi-Perf DBAT is waiting for the next client UOW If Hi-Perf DBAT has not received new work for POOLINAC time > DBAT will be purged and the connection will go inactive If # of Hi-Perf DBATs exceed 50% of MAXDBAT threshold > DBATs will be pooled at commit and package resources copied/allocated as RELEASE(COMMIT) Hi-Perf DBATs can be purged to allow DDL, BIND, and utilities to break in > Via -MODIFY DDF PKGREL(COMMIT) 31 Performance and Scalability DBM1 31-bit Virtual Storage Constraint Relief with 64-bit SQL run time More persistent threads with RELEASE(DEALLOCATE) is also trade off with BIND/REBIND and DDL concurrency For RELEASE(DEALLOCATE) some locks are held beyond commit until thread termination Mass delete locks (SQL DELETE without WHERE clause) Gross level lock acquired on behalf of a SQL LOCK TABLE Note: no longer a problem for gross level l lock acquired by lock escalation CICS-DB2 accounting for cost of thread create and terminate, or avoidance thereof CICS uses the L8 TCB to access DB2 irrespective of whether the application is thread safe or not Thread create and terminate cost will clock against the L8 TCB and will be in the CICS SMF Type 110 record Note: prior to OTE did not capture the thread create in the SMF Type Performance and Scalability DBM1 31-bit Virtual Storage Constraint Relief with 64-bit SQL run time High Performance DBATs (Hi-Perf DBATs) should be carefully Want to have some high performance applications running on LUW application servers connected to DB2 10 for z/os running with High Performance DBATs and others not Standard ODBC and JDBC packages supplied with drivers/connect packages should be bound twice into two different package collections e.g., The CS package in collection1 will be bound with RELEASE(DEALLOCATE) so that the applications using that package will be eligible to use high performance DBATs The CS package in collection2 (e.g., NULLID) would be bound with RELEASE(COMMIT) and would not use high performance DBATs For JDBC applications Set the currentpackageset property in the respective datasource For.NET and ODBC / CLI applications Set CurrentPackageSet parameter in the db2dsdriver.cfg configuration 32

17 Performance and Scalability DBM1 31-bit Virtual Storage Constraint Relief with 64-bit SQL run time Potential to reduce the number of DB2 subsystems in data sharing group First step is to collapse multiple DB2 members running on the same LPAR May then be able to reduce the number of LPARs/DB2 members Consider the increase in logging rate per DB2 member Possible aggravation of LC19 contention despite V10 improvement Consider the increase in SMF data volume per LPAR Can enable DB2 compression of SMF data to reduce SMF data volume > Experience is that Accounting records compress 70-80% > Tiny CPU overhead at ~1% Re-consider use of accounting roll up for DDF and RRSAF workload (default) > Compromises performance PD/PSI as lose information on outlying transactions > Significant enhancements to package level accounting so it is now useful Consider the increased DUMPSRV and MAXSPACE requirement Re-emphasise the continued value of data sharing to differentiate the platform Support avoidance of planned outages Avoid humongous single points of failure Minimum of 4-way for true continuous availability 33 Performance and Scalability 31-bit and 64-bit virtual storage contraction CONTSTOR=YES and MINSTOR=YES These existing system parameters drive the contraction of 31-bit storage pools and the best fit allocation of 31-bit storage respectively Not applicable to 64-bit storage Not as critical as before V10 Assuming generous DBM1 31-bit VSCR in V10, set CONSTOR=MINSTOR=NO 64-bit thread pools are contracted under control of Commit count New Real Storage Management DISCARD function (see follow on slides) 35 Performance and Scalability 64-bit virtual storage Three large areas allocated at IPL time Common 6GB (z/os default) Addressable by all authorized programs on the LPAR IFC for accounting Private 1TB Buffer pools XML and LOB are huge users, RTS blocks, TRACE buffers, some RID blocks, IFC work buffers and few other misc system pools Shared (Private) 128GB Addressable by all authorized products which have registered their interest to z/os using the unique object token created when the memory object is created V9 introduced 64-bit shared private storage but it was used in limited fashion Almost all the DB2 storage in V10 is now 64-bit shared private DB2 is only "reserving" virtual storage, it does not mean it is being used It costs nothing to reserve virtual storage i.e., addressing range Having a fixed size areas is a lazy design but it makes it easier for serialization Needs to be backed by real storage when it is allocated within the reference area 34 Performance and Scalability Real storage Need to carefully plan, provision and monitor real storage consumption Prior to V10 a hidden zparm SPRMRSMX ( real storage kill switch ) existed SPRMRSMX prevents a runaway DB2 subsystem from taking the LPAR down Should be used when there is more than one DB2 subsystem running on the same LPAR Aim is to prevent multiple outages being caused by a single DB2 subsystem outage Should to set to 1.5x to 2x normal DB2 subsystem usage Kills the DB2 subsystem when SPRMRSMX value reached With V10, will need to factor in 64-bit shared and common use to establish new footprint Problems with introduction of V10 Unable to monitor the REAL and AUX storage frames used for 64-bit shared storage V9 not really an issue, as limited use of 64-bit shared But now V10 makes extensive use of 64-bit shared LPAR level instrumentation buckets for REAL and AUX storage use If more the one DB2 subsystem on the same LPAR then the numbers reported are inaccurate Only able to get reliable numbers if only one subsystem like DB2 on the LPAR uses 64-bit shared Lack of ENF 55 condition monitoring 50% of AUX used 36

18 Performance and Scalability Performance and Scalability Real storage DB2 APAR PM24723 is very important Monitoring issue is addressed and new extensions to IFCID 225 provided Pre-req is new MVS APAR OA35885 which provides a new callable service to RSM to provide REAL and AUX used for addressing range for shared objects SPRMRSMX hidden zparm now becomes an opaque parameter REALSTORAGE_MAX Introduces DISCARD mode to contract storage usage to protect against excessive paging and use of AUX New zparm REALSTORAGE_MANAGEMENT controls when DB2 frees storage frames back to z/os > ON -> Discard unused frames all the time - discard stack, thread storage, keep footprint small > OFF -> Do not discard unused frames unless things are getting out of hand > AUTO (default) -> Detect whether paging is imminent and reduce the frame counts to avoid system paging With AUTO, DB2 monitors paging rates, switches between ON/OFF and decides when to discard frames based on > 80% of SPRMRSMX reached > 50% of AUX (ENF55 condition) used > Hitting AVQLOW (available real storage frame) New messages (DSNV516I, 517I) for when paging rate thresholds cause DB2 to free real frames Strong recommendation to apply PTF for APAR PM24723 before going into business production and to run with REALSTORAGE_MANAGEMENT=AUTO Monitoring Virtual and Release Storage SPREADSHEETDD support in OMPE has not been enhanced to support V10 OMPE are working on a 'generic' spreadsheet generator Outstanding requirement to also include serviceability fields MEMU2 and MEMUSAGE already enhanced for V10 and available on the DB2 for z/os Exchange community website on IBM My developerworks 1.From IBM My developerworks My Home (sign in with your IBM login at search 'memu2' in All My developerworks. 2.(From DB2 for z/os Exchange ( click on 'View and download examples'. The file is tagged with 'memu2'. 3.To access MEMU2 directly (but note that if you want to be kept informed of updates and new versions, you need to log on to developerworks rather than download the file anonymously...) V8/V9 V Questions 39 40

19 IBM DB2 10 Migration Planning and Very Early Experiences Part 2 John Campbell Distinguished i Engineer DB2 for z/os Development Notes: John Campbell1/UK/IBM@IBMGB CampbelJ@uk.ibm.com 1 Agenda Keys to customer migration success Performance and Scalability BIND, REBIND and EXPLAIN Availability Migration Planning Removal of DDF Private Protocol Security Considerations when removing DDF Private Protocol Other Summary 3 Objectives Share lessons learned, surprises, pitfalls Provide hints and tips Address some myths Provide additional planning information Provide usage guidelines and positioning on new enhancements 2 Performance and Scalability High INSERT performance Significant improvements for UTS Now support for MEMBER CLUSTER Changes to space search algorithm (like classic partitioned) Goal was for UTS to be equal or better than classic partitioned (PTS) Not there yet, but much closer Very workload dependent Some good, some worse Still trade off between space vs. throughput and reduced contention Work still to do on UTS PBR/PBG with RLL and sequential insert 4

20 Performance and Scalability High INSERT performance Reduced LRSN spin for inserts to the same page Works well for MRI and INSERT within loop in a data sharing environment Optimization for pocket sequential insert works well Index manager picks the candidate RID during sequential insert (next lowest key rid) Higher chance to find the space and avoiding a space search Parallel index IO works very well when activated for random key inserts >= 3 indexes Prefetch and deferred write offload to ziip to compensate 5 DB2 10 Range Defined Table Spaces Random Inserts Throughput CPU Time PLL RLL PLL RLL PTS PTS/MC PBR PBR/MC PTS PTS/MC PBR PBR/MC High Insert Workload Description 2-way data sharing Database schema 3 tables with total of 6 indexes (4 unique, 2 non-unique indexes, 2 secondary indexes) Table space types: Classic Partitioned, Classic Segmented, UTS (PBR, PBG) SQL INSERTs contain 5, 9 and 46 columns of integer, bigint, char, varchar, decimal and timestamp data type Application implemented in Java Sequential inserts into empty tables 240 concurrent threads Multi-row inserts (100) Random inserts into populated tables 200 concurrent threads Single-row inserts 6 Rows/Sec Msec / Commit Sequential Inserts Throughput CPU Time PLL RLL PTS PTS/MC PBR PBR/MC PLL RLL PTS PTS/MC PBR PBR/MC Rows/Sec Msec / Commit 7 DB2 10 Non-range Defined Table Spaces Random Inserts Throughput CPU Time Rows/Sec PLL RLL Msec / Commit 0 PLL RLL SEG PBG PBG/MC Sequential Inserts SEG PBG PBG/MC SEG PBG PBG/MC Throughput CPU Time PLL RLL SEG PBG PBG/MC PLL RLL SEG PBG PBG/MC Rows/Sec Msec / Commit 8

21 Performance and Scalability Accounting Trace Class 3 enhancement separate counters IRLM Lock/Latch waits DB2 Latch waits Data sharing Faster DB2 shut down by avoiding local buffer pool scan per GBP-dependent object Avoiding scan of XXXL local Buffer pool when Pageset/partition transitions into GBP-dependency Pageset/partition transitions out of GBP-dependency Inline LOBs work very well if you hit the sweet spot Potential for significant CPU and elapsed time improvement with the right inline value Trade off in setting the right inline value Avoiding access to auxiliary tablespace Increasing base row size with fewer rows per page May have to increased page size Inline portion can be compressed Significant space savings with small LOBs (<1000 bytes) 9 Performance and Scalability Hash access vs. Index only access and index lookaside Competes against index only access and index lookaside Advantage that index only access still provides for clustered data access Can now have unique index with INCLUDE columns Reduce number of indexes required for performance reasons Improve insert, update and delete performance Need to find the sweet spot High NLEVELS in index (>=3) Purely direct row access by primary key Truly random access Read intensive, not volatile No range queries Many rows per page etc Space allocation of fixed hash space is key to control overflow Too small will lead to rows in overflow Too large will lead to random IO REORG AUTOESTSPACE(YES) but still some rows in overflow Degraded LOAD and REORG utility performance 11 Performance and Scalability Compress on INSERT Compression ratios almost as good compared with running REORG later Active log writes Prior to V10, log writes are done serially when re-writing partial CIs Determined that destructive writes due to IO errors no longer occur Now all log write IOs are done in parallel Elapsed time improvements Limited value of Currently Committed locking semantics SQL UPDATE not supported 10 Performance and Scalability Improved index space search when index leaf page split In V8/V9, Index Manager First searches the space map page covering the splitting page If there is no free entry, searches the space map pages starting from the first space map page to the highest allocated page If all the space map pages are full, has to extend If the index is huge and all the space map pages having free entries are toward the end of the index, this process can take a very long time In V10, Index Manager After searching the space map page covering the splitting page, and if it is full, will start searching from the page number it last remembered having the free entry in it (page A) When it reaches to the highest allocated page, it starts from the beginning and search forward till it reaches to page A Then it has to extend since the entire index is full The page number of the space map page having free entry is stored in an in-memory control block When Index Manager finds a space map page with free entry, it is updated to be the page number of that space map page Index Manager updates this value when an index page is deleted or when the index is mass deleted Retrofitted back to V9 via APAR PM

22 Performance and Scalability Comparing CPU performance on V10 relative to V8 or V9 More difficult to do in real customer production environment Uncertainty caused by application changes Fluctuation in the daily application profile especially batch flow Must try to normalise things out to ensure workloads are broadly comparable Broadly similar in terms of SQL and getpage profile Usually have to exclude the batch flow Factor out extreme variation Need to look at multiple data points 13 BIND, REBIND and EXPLAIN Access Path Reuse - APREUSE and APCOMPARE Introduced in V10 through APARs PM33767 (function) and PM25679 (enabled) Works independently of access plan stability (PLANMGMT=EXTENDED BASIC) Provides ability to generate new SQL run time but will attempt to reuse the existing access path by internally using OPTHINTS Applies to both BIND REPLACE and REBIND Available in CM when migrating from V9 -> V10 and then applies to subsequent BIND/REBIND in V10 Only works when package previously bound on V9 or higher Starting with V9, EXPLAIN information is saved away inside the package in SPT01 Referred to as Explain Data Block (EDB) which is a compact representation of PLAN_TABLE information Not guaranteed to succeed in all cases as OPTHINTS are not enforceable 100% of the time Some incompatibilities between old and new release (restrictions) Seeing 95-99% compatibility 15 BIND, REBIND and EXPLAIN Value of REBIND under V10 Improved performance from new run time (avoid puffing, enable SPROC) Maximize DBM1 31-bit VSCR Allow RID overflow to workfile Take advantage of query optimization changes (available in CM mode) Reduce exposure to problems with migrated packages from earlier releases INCORROUTs Thread abends Can mitigate exposure to bad access path change introduced with REBIND which leads to degraded run time performance (regression) Use access plan stability (PLANMGMT=EXTENDED BASIC) and fallback if needed PLANMGMT=EXTENDED is now the default Use APREUSE and APCOMPARE 14 BIND, REBIND and EXPLAIN Access Path Reuse - APREUSE and APCOMPARE Conservative approach where minimal access path changes are required Step 0 (Optional) Use REBIND... EXPLAIN(ONLY) + APREUSE(ERROR) Perform an impact analysis before actual REBINDs Step 1: REBIND PACKAGE (*) Use PLANMGMT(EXTENDED)... backup of V9 access paths, just in case + EXPLAIN(YES) + APREUSE(ERROR) Step 2: For packages that failed Step 1 (i.e., leftovers) 2a: Leave them as is... they will be at the old level OR 2b. REBIND with PLANMGMT(EXTENDED) + APREUSE(NO) > This step exposes yourself to access path changes > But you have a backup 16

23 BIND, REBIND and EXPLAIN RUNSTATS/REBIND recommendations based upon on scenario V8 preparation If RUNSTATS will be difficult on large number of objects immediately after migration to V9/10, then REORG and/or RUNSTATS (V8) immediately prior to migration can reduce RUNSTATS need on V9/10 - as RUNSTATS INDEX under V10 can be sufficient to capture new CR/DRF V9 migration RUNSTATS objects as soon as possible after migration Target dynamic applications first as these are exposed to new access paths immediately Delay static REBINDs until associated objects have RUNSTATS run V8->V10 migration RUNSTATS objects as soon as possible after migration Target dynamic applications first as these are exposed to new access paths immediately Equal priority - target static parallelism packages to REBIND to avoid incremental bind at each execution Delay non-parallelism REBINDs until associated objects have RUNSTATS run V9->V10 migration REBIND static parallelism packages as soon as possible to avoid incremental bind at each execution Delay non-parallelism REBINDs until associated objects have RUNSTATS run BIND/REBIND options APREUSE/APCOMPARE are available on V10 for packages bound on V9 17 BIND, REBIND and EXPLAIN RUNSTATS/REBIND recommendations based upon on scenario V9/10 co-existence Set ABIND=COEXIST while in co-existence with V8 What to do with static parallel queries? Accept incremental bind whenever executed on V10 member OR, REBIND with DEGREE('1') to disable parallelism while in co-existence. Follow V8-V10 migration steps after all members are V10, including resetting the following zparms Set ABIND=YES 19 BIND, REBIND and EXPLAIN RUNSTATS/REBIND recommendations based upon on scenario V8/9 co-existence Set STATCLUS=STANDARD while in co-existence with V8 Set ABIND=COEXIST while in co-existence with V8 Avoid REBIND while in co-existence with V8 Follow V9 migration steps after all members are V9, including resetting the following zparms Set STATCLUS=ENHANCED Set ABIND=YES V8/10 co-existence Set STATCLUS=STANDARD while in co-existence with V8 Set ABIND=COEXIST while in co-existence with V8 What to do with static parallel queries? Accept incremental bind whenever executed on V10 member OR, REBIND with DEGREE('1') to disable parallelism while in co-existence. Follow V8-V10 migration steps after all members are V10, including resetting the following zparms Set STATCLUS=ENHANCED Set ABIND=YES 18 BIND, REBIND and EXPLAIN Single thread BIND/REBIND performance Degraded CPU and elapsed time performance on entry to CM PLANMGMT=EXTENDED is now default New indexes defined for post ENFM when hash links are eliminated Change in access path (index access) on entry to CM No concurrency improvement until after Catalog restructure in ENFM Concurrent BIND/REBIND performance Problems addressed Performance problems related to DELETE/INSERT process Space growth in SPT01 for both LOB space and base table Now working well Inefficient space search for out of line LOB in data sharing (APAR PM24721) Inline LOB with compression for SPT01 to address SPT01 growth (APAR PM27073) More efficient space reuse for base table and UTS (APAR PM27973) Recommendations Customers need to change existing procedures to go parallel But cannot do this until post ENFM Benefit from reducing application down time to implement new application releases 20

24 BIND, REBIND and EXPLAIN EXPLAIN tables Format and CCSID from previous releases is deprecated in V10 Cannot use pre V8 format SQLCODE V8 or V9 format Warning SQLCODE regardless of CCSID EBCDIC or UNICODE Must not use CCSID EBCDIC with V10 format EXPLAIN fails with RC=8 DSNT408I SQLCODE = -878 BIND with EXPLAIN fails with RC=8 DSNX200I Recommendations Use CCSID UNICODE in all supported releases (V8, V9, V10) due to problems with character truncation and conversion etc Use the V10 extended column format with CCSID UNICODE when Applications access EXPLAIN tables and can only tolerate SQLCODE 0 or +100 V10 column format is supported under V8 and V9 with the SPE fallback APAR PK85956 applied with the exception of DSN_STATEMENT_CACHE_TABLE due to the BIGINT columns APAR PK85068 can help migrate V8 or V9 format to the new V10 format with CCSID UNICODE 21 Availability Online Schema Evolution ( Deferred Alter ) Migrate from classic table space types (simple, segmented, partitioned) to UTS PBG/PBR One way ticket only UTS is pre-requisite for Cloned Table, Hash, Inline LOB, Currently Committed Once migrated to UTS PBG/PBR can change attributes in both directions DSSIZE, index page size, MEMBER CLUSTER, Hash Access, Benefits Streamed line way to move to UTS Reduce administrative time and cost Cuts down on errors Reduce outages Issue that PIT recovery to point before successful materializing REORG not possible Incorrect results from REORG Application change rollback 23 Capturing documentation for IBM Methods for capturing documentation for all releases is documented here OSC and DB2PLI8 do not support DB2 10 SYSPROC.ADMIN_INFO_SQL supports V8 -> V10 (Required) Excellent developerworks article here: capturequery/index.html It is installed in V10 base and is subject to the installation verification process DB2HLQ.SDSNSAMP(DSNTESR) will create and bind it calling program is DSNADMSB, and sample JCL in DSNTEJ6I Ensure DB2 9 and DB2 10 have APAR PM39871 applied Data Studio V3.1 incorporates this procedure into a GUI (Best Practice) No charge product, replacement for OSC and Visual Explain Several versions: DBA s should download the Administration Client Incorporates Statistics Advisor FTP doc directly to DB2 Level 2 Can be used to duplicate stats in TEST environment 22 Availability Restart Light enhancement LBACKOUT will now be honoured LBACKOUT=YES AUTO will cause postponed abort (PA) URs to be created Restart will complete DB2 will shut down Retained locks will be kept on behalf of PA URs Controlled via new system parameter Also retrofitted back to V9 via APAR Online REORG with FORCE Only running threads which are blocking are cancelled Threads which are suspended / inactive will cause REORG to still fail Online REORG LOB with DISCARD Cannot handle LOB columns greater than 32KB 24

25 Migration and Planning Migration process very similar to V8 and V9 Works well with few problems with migration fallback Migration from either DB2 for z/os V8 NFM or DB2 9 for z/os NFM These migration fallback sequences are not valid V8 NFM > V10 CM8 > V8 NFM > V9 CM V8 NFM > V9 CM > V8 NFM > V10 CM8 Fallback Toleration SPE APAR PK56922 Early Code For V8/V9 APAR PK87280 (superseeds APAR PK61766) Information APARs II14474: V8 to V10 II14477: V9 to V10 25 Migration and Planning Use of V10 Early Code with V8 It will take an IPL to originally install the V10 Early Code V8 Early Code does not understand REFRESH However, subsequent maintenance to the V10 Early Code can be accomplished with a -REFRESH command If coming from V8 BSDS must be reformatted for larger active / archive tracking IPL amounts for need to be adjusted based on number of DB2 members 64-bit Private (1TB) 64-bit Shared (128GB) 64-bit Common (6GB) 27 Migration and Planning V8 premigration checklist: V8 migration checklist: V9 premigration checklist: V9 migration checklist: 26 Migration and Planning DB2 Connect Minimum level V9.5 FP7 V9.7 FP3A and for new functions Start with the latest levels based on CST/RSU and stabilise 28

26 Migration and Planning DBRMs bound directly into plans no longer supported If found in V10, will trigger auto bind into packages For V8 and V9 APARs PK62876/PK79925 adds new syntax to convert from DBRMs to packages REBIND PLAN option COLLID Could result in access path change APARs PM01821 (Version) and PM30382 (Location from * to blank) should be on Best to migrate DBRMs to packages before migrating to V10 Old plans and packages bound prior to V6 will require REBIND Catalog and Directory must be SMS managed (EF, EA) ahead of CM PDSEs required for SDSNLOAD, SDSNLOD2, ADSNLOAD DSNHDECP NEWFUN=V10 V9 V8 29 Removal of DDF Private Protocol Must absolutely eliminate all use of DDF Private Protocol before migrating APAR PK92339 in V8 introduces new zparm PRIVATE_PROTOCOL=YES NO Must be in sync with zparm setting of DBPROTCL DSNT225I message to indicate private protocol failure for REBIND IFCID 157 can be used to identify packages which are using private protocol Beware APAR PM17665 removes authorization behaviour for private protocol i.e.,. Execute privilege on package is required for the user running the program under DRDA APAR PM37300 will provide relief (see later slides) APAR PK64045 (PE, PM14816) in V8 new zparm DRDA_RESOLVE_ALIAS Private protocol always performs alias resolution 3-part name is sent to the server If DRDA_RESOLVE_ALIAS is set to YES then also DRDA connections get this resolution e.g., For remote BIND with COPY option 31 Removal of DDF Private Protocol Must absolutely eliminate all use of DDF Private Protocol before migrating No longer supported In V10 Any local packages miss tagged with DDF Private Protocol will be tolerated Otherwise package must exist in both local and remote sites A lot of packages and plans are bound with DBPROTOCOL(PRIVATE) because this was the default (zparm DBPROTCL) when introduced in DB2 V6 DSNT226I is issued if DBPROTOCOL(PRIVATE) is used during REBIND 30 Security considerations when removing DDF Private Protocol This section also applies to customers using DRDA exclusively There are fundamental differences on how authorization is performed based on the distributed protocol used Private Protocol (DB2 for z/os requester) Supports static SQL statements only Plan owner must have authorization to execute all SQL executed on the DB2 server Plan owner authenticated on DB2 requester and not on the DB2 server DRDA Protocol Supports both static and dynamic SQL statements Primary auth ID and associated secondary auth IDs must have authorization to execute package and dynamic SQL on the DB2 server Primary auth ID authenticated and secondary auth IDs are associated on DB2 server Prior to V10, Private Protocol and DRDA Protocol can be used by same application Private Protocol security semantics was used due to possible inconsistent behavior which is dependent on how programs are coded and executed 32

27 Security considerations when removing DDF Private Protocol But there is also a difference prior to V10 in the authorizations required by an incoming DRDA connection at the DB2 for z/os server, depending on where the connection come from: Dynamic SQL DRDA connection from DB2 Connect and/or DB2 client direct connection Connecting userid needs authority to run the appropriate DB2 package and authority to access the DB2 table Dynamic SQL DRDA connection from DB2 for z/os requester Connecting userid needs authority to access the DB2 table Originating plan owner needs authority to run the appropriate DB2 package It is different for DB2 for z/os requester to DB2 for z/os server because connections were designed to use Private Protocol (PP) semantics to avoid changing authids when switching between PP to DRDA Protocol With the disappearance of PP in V10, DB2 have decided to bring the DRDA connection from DB2 for z/os requester to DB2 for z/os server in line with other DRDA requesters and to change the authorizations required This was retrofitted back into V8 and V9 with APAR PM17665 It is very important to distinguish clearly between the behavior of DRDA before and after APAR PM Security considerations when removing DDF Private Protocol In summary Before disabling private protocol, ensure all appropriate grants are performed Grant execute privilege to any user who plans to run a package or stored procedure package from a DB2 for z/os requester, just like other DRDA clients DB2 V8 and V9 can disable private protocol but still maintain private protocol authorization checks by Setting system parameter PRIVATE_PROTOCOL=AUTH DB2 10 does not support private protocol but can allow private protocol authorization checks for use of DRDA protocol for DB2 for z/os requesters by Setting system parameter PRIVATE_PROTOCOL=AUTH 35 Security considerations when removing DDF Private Protocol APAR PK92339 introduced new zparm PRIVATE_PROTOCOL=YES NO To prevent future introduction of PP then set PRIVATE_PROTOCOL=NO The result of migrating to V10 or the introduction of APAR PM17665 under V8 or V9, when running with PRIVATE_PROTOCOL=NO introduces the authorization changes at the DB2 for z/os server for DRDA connections coming from DB2 for z/os requester PP security semantics are no longer used as default for access from a DB2 for z/os requester Plan owner value is ignored and connecting userid must be granted authority to execute the package at the remote site Otherwise the connection will fail with SQLCODE -551 As a result of customer complaints, APAR PM37300 introduces PRIVATE_PROTOCOL=AUTH which allows an installation to Disable PP but keep the plan owner authorization check (the "private protocol semantics ) Migration to V10 or the application of PTF for APAR PM17665 does affect you even if you have everything already bound as DRDA 34 Other Ability to create classic partitioned table space (PTS) Classic PTS deprecated in V10 By default will be created as UTS PBR UTS will only support table based controlled partition syntax Options to be able to officially create classic PTS Specify SEGSIZE 0 on CREATE TABLESPACE Set new zparm DPSEGSZ=0 (default 32) Fast Log Apply storage System parameter LOGAPSTG is eliminated Total FLA storage is now 510MB Old COBOL and PL/1 V7 lookalike pre-compiler (DSNHPC7) for older COBOL and PL/I is still provided DDL Concurrency after Catalog restructure Some help provided but concurrency issues not absolutely solved Still deadlocks with parallel heavy DDL against different databases 36

28 Other SPT01 compression is back Via system parameter COMPRESS_SPT01=YES (default is NO) Statistics Interval Only the frequency of IFCIDs 105, 106, 199 are controlled via STATIME IFCIDs 2, 202, 217, 225, 230 are always cut at a fixed 1 minute interval Remember to normalise the data when comparing V8/V9 vs. V10 37 Other Incompatible changes CHAR() scalar function Solution via APAR PM29124 to restore compatible behavior for CHAR SELECT COL1,CHAR(COL1),HEX(CHAR(COL1)),SUBSTR(CHAR(COL1), 2, 4) APPL_SUBSTR FROM DB2_TABLE; ! COL1! CHAR_V9! HEX_CHAR_V9! APPL_SUBSTR!! COL1! CHAR_V10! HEX_CHAR_V10! APPL_SUBSTR! ! 9! 0009,! 40F0F0F0F96B! 0009!! 9! 9! F !!! 99! 0099,! 40F0F0F9F96B! 0099!! 99! 99! F9F ! 9!! 999! 0999,! 40F0F9F9F96B! 0999!! 999! 999! F9F9F ! 99!! 9999! 9999,! 40F9F9F9F96B! 9999!! 9999! 9999! F9F9F9F94040! 999! Other RUNSTATS Page Sampling Performance improvement can be phenomenal Potential issues with accuracy because error rates increase as the sample size decreases No sampling done on indexes ziip offload Nearly all RUNSTATS INDEX processing is offloaded, but only basic RUNSTATS TABLE processing is offloaded Much less for advanced/complex statistics Not supported: > Inline stats > COLGROUP > DSTATS > Histogram stats 38 Other Incompatible changes Create Trigger Impact: Trigger created in V10 not executable in V9 Workaround: drop and create trigger in V9 PMR open DSNT408I SQLCODE = -723, ERROR: AN ERROR OCCURRED IN A TRIGGERED SQL STATEMENT IN TRIGGER AWAT.OCA_TEST_TRIGGER. INFORMATION RETURNED: SQLCODE - 904, SQLSTATE 57011, MESSAGE TOKENS 00E30305, ,AWAT.OCA_TEST_TRIGGER-.18E5950B04A23EEC, SECTION NUMBER 1 40

29 Other Incompatible changes SQL stored procedures SQLCODE = -904, ERROR: UNSUCCESSFUL EXECUTION CAUSED BY AN UNAVAILABLE RESOURCE. REASON 00E7009E, TYPE OF RESOURCE , AND RESOURCE NAME BE2TFKT.LESE_VARIANTEN.18DEB14D119115C4 Impact: Deployment has to be executed on V9 member in V10 Workaround: run ALTER PROCEDURE REGENERATE on V9 member APAR PM Summary Very good release in terms of the opportunities for price/performance and scalability improvement Significant DBM1 31-bit VSCR after rebind Use long term page fixed buffer pools Exploit 1MB real storage page frames on z10 and z196 Reduced latch contention, log manager improvements, etc Opportunity for further price performance improvements More use of persistent threads CICS, IMS/TM, High Performance DBATs More use of RELEASE(DEALLOCATE) with persistent threads More use of RELEASE(DEALLOCATE) is a trade off Increased storage consumption > Need to plan on additional real memory Reduced concurrency > BIND/REBIND and DDL Increase MAXKEEPD to reduce short prepares for dynamic SQL Opportunity for scale up and LPAR/DB2 consolidation 43 Other DB2 for z/os Exchange on DeveloperWorks Relaunch and rebranding of ' DB2 zhotline' It is a DB2 Technical Newsletter for DBAs and Systems Programmers Critical APAR info (more than HIPER, less than Red Alert) Living undocumented incompatible change list Critical issues seen in field and ways to navigate Hints, tips, practical advice Will be updated monthly RSS Feed from DeveloperWorks into IDUG Link nityuuid=22586cb d2c-ae74-0ddcc2a409bc 42 Summary Carefully plan, provision and monitor real storage consumption Any customer migrating from either V8 or V9 to V10 should make a solid plan, take extra care to mitigate against the risk and set themselves up for success Regular full major maintenance drops Exploitation of CST/RSU recommended maintenance Augment by regular use of Enhanced HOLDDATA Perform application regression and stress testing to keep fires away from production Plan should allow some contingency for some bumps in the road 44

30 Questions 45 46

31 SQLADRIA SEMINAR Ljubljana, 13 April 2012 Advanced Backup and Recovery using Flashcopy? V10 Virtual and Real Storage Management? John Campbell IBM Sponsored by

32

33 DB2 for z/os Advanced Backup and Recovery using Flashcopy John Campbell Distinguished Engineer DB2 for z/os Development Agenda DB2 Managed FlashCopy Solution in V8 DB2 System Level Backup using Copy Pools Backup System and Restore System Utilities Backup and Recovery Enhancements in DB2 9 Automatically manage Copy Pool backups to tapes Allow table space recovery using System Level Backups Incremental FlashCopy Backup and Recovery Enhancements in DB2 10 Copy Pool Enhancements in z/os 1.12 Disclaimer Copyright IBM Corporation All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE IBM, the IBM logo, ibm.com, and DB2 are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at 2 DB2 Managed FlashCopy Solution Provide an easier and less disruptive way for fast volume-level backup and recovery Use FlashCopy to backup DB2 data and logs No longer need to suspend logs Backups are managed by DB2 and DFSMShsm to support system level PIT recovery, Disaster Recovery and Cloning DBs Logs DBs Logs FlashCopy BSDSs ICFs BSDSs ICFs

34 DB2 Managed FlashCopy Solution... New utilities in DB2 for z/os V8: BACKUP SYSTEM RESTORE SYSTEM Takes system-level copies of data and logs Exploits SMS Copy Pool DB2 data and logs must be SMS-managed Write log activity is NOT suspended Suspends data set creation, deletion, rename, and extend operations SMS COPYPOOL - Example Name: DSN$P870$DB Versions: 2 CopyPool CopyPool Name: DSN$P870$LG Versions: 2 Storage Group Storage Group Storage Storage Group Group Type: Pool Type: Pool Type: Type: Extend Pool Name: DSNCAT Name: DSNDB Name: Name: EX1 DSNLOG CopyPool Backup Name: CPB1 CopyPool Backup Name: CPB1 CopyPool CopyPool Backup Backup Name: Name: CPB2 CPB1 Type: CopyPool Backup Name: CPB1 CopyPool Backup Name: N/A Storage Group SMS COPYPOOL SMS construct Set of SMS storage groups - maximum 256 Has a VERSIONS attribute - maximum 85 Each DB2 system has two SMS COPYPOOLs DATA COPYPOOL (DSN$location_name$DB) LOG COPYPOOL (DSN$location_name$LG) name$lg) Copy Pool Backup New storage group type Used to hold volume copies of DASD defined in the COPYPOOL Backup and Recovery Enhancements in DB2 9

35 DB2 9 Enhancements BACKUP SYSTEM and RESTORE SYSTEM utilities will manage Copy Pool backups to/from tapes Up to five tape copies can be created Support Incremental FlashCopy RECOVER utility can use Copy Pool backups as the source for recovery of DB2 tables and indexes Backups can be on DASDs or Tapes Prereq. DFSMShsm and z/os 1.8 RESTORE SYSTEM Tape Support Restore the DB copy pool from tapes in parallel If the data copy pool backup resides on DASD and on tape, then the DASD version is chosen as the recovery base Install ZPARM options FROMDUMP - user can specify that they don't want to use the DASD version DUMPCLASS (dc) - user can specify a specify HSM dump class to restore from CopyPool Backups on Tapes Customer benefits: This sure makes Copy Pool dump & restore management a lot easier Reduce disk space for maintaining multiple Copy Pool backups Integrated tape management between DB2 and DFSMShsm Retaining Copy Pool backups for long term use Providing a means of recovery from media failure Remote site recovery RECOVER utility use Copy Pool backup RECOVER utility enhancements permit using a backup taken at the systemlevel as the recovery base for a subset of objects in the system Need to set ZPARM System_Level_Backups = YES Connecting system-level backups with object level recoveries FULL or DATA TS 1 BACKUP SYSTEM System Level Backup RECOVER FULL or DATA TS 2

36 RECOVER utility use Copy Pool backup... Most recent recovery base (prior to the recovery point) is chosen: Could be image copy, concurrent copy, log yes event, or Copy Pool backup Takes sub-second to restore a data set if the backup is on DASD (independent of its size) Use normal I/O (i.e. not FlashCopy) If FlashCopy background copy is not complete If the production volume is the source of PPRC/XRC Use FlashCopy with Remote Pair Flash Copy on DS8K 4.2 Make sure SETSYS FASTREPLICATION(PREFERRED) is used, which is the default When restoring a list of objects, the restore process is done in parallel Incremental Flash Copy Introduced by DFSMShsm in z/os 1.8 Initial incremental FlashCopy creates full base backup Subsequent incremental FlashCopies copy changed tracks to backup volumes only (overriding initial backup) Minimizes I/O impact (DB2 incremental IC) Considerably reduces elapsed time of physical copy RECOVER utility use Copy Pool backup... Data set must be cataloged and allocated on the same volumes that it resided on at the time of the backup Support for data sets that have extended to new volumes DB2 Recovery Expert V2 can alleviate this problem DFSMS support is in z/os 1.11 (require DB2 apar PK92725) SMS option to capture ICFCTLG for Copy Pool Allow recovery for moved/deleted data sets Still need to have sufficient free space on the original DASD volume If the restore of datasets from DASD fails, then the recovery of the object will not proceed Use the RECOVER RESTOREBEFORE option to direct the utility to use a recovery base prior to the system-level backup If FROMDUMP is specified: Data sets are restored from tapes RECOVER to PIT with consistency DB2 RECOVER utility will: Automatically detect the uncommitted transactions running at the recover PIT Roll back their changes on the recovered objects. Thus ensuring data consistency after PIT recoveries. Recovered objects left in a transaction consistent state Avoid the need to regularly run the QUIESCE utility Reduces disruption to DB2 users and applications

37 Backup and Recovery Enhancements in DB2 10 V10: COPY Dataset-level Flashcopy support COPY, RECOVER, REORG, LOAD, REBUILD INDEX, REORG INDEX New zparms & utility parms to govern Significant CPU & elapsed time reduction for large pagesets Create transaction-consistent image copies from SHRLEVEL CHANGE CPU time per object (z10) Elapsed time per object (z10) IC FC Object size (MB) DB2 10: Copy/Recover Enhancements Improved COPY CHANGELIMIT performance Use RTS instead of scanning space map pages Dataset level FlashCopy option Inline COPY is optional for REORG (post-ga enhancement) FlashCopy backups with consistency and no application outage FlashCopy backups as input to: RECOVER (fast restore phase) COPYTOCOPY Create sequential copies from FlashCopy Can be used as input for UNLOAD DSN1COPY/DSN1PRNT RECOVER back to log point REPORT RECOVERY support for system level backups MODIFY RECOVERY improved performance 18 Seconds Seconds Object size (MB) IC FC V10: COPY Post-GA Flashcopy enhancements PM34776 Support for REORG AUX YES to FlashCopy aux objects Remove requirement for sequential inline copy for REORG Add a new zparm parameter FLASHCOPY_PPRC which controls whether FlashCopy can be taken against PPRC primary volumes REQUIRED - Require DS8K Remote Pair FlashCopy function to take FlashCopy PREFERRED - Allow to take FlashCopy even if DS8K RPFC is not available (volume pairs will be in GDPS duplex pending state when the background copy is in progress) NONE - FlashCopy does not preserve mirror CPU time per object (z10) Elapsed time per object (z10) IC FC Seconds IC FC Seconds Object size (MB) Object size (MB)

38 V10: COPY Improved dataset management & performance CHANGELIMIT will not allocate copy dataset unless copy taken &ICTYPE now matches actual image copy type Use RTS for CHANGELIMIT performance No longer need to scan space map pages Incremental copy will not allocate copy dataset unless pages changed Insert dummy SYSCOPY record for incremental copy even though no pages changed Delivery post-ga PPRC/XRC Restrictions RESTORE SYSTEM & RECOVER from SLB fails if target is a primary in a PPRC relationship or part of an XRC relationship DFSMShsm APAR OA23849 permits FlashCopy to a PPRC primary But volumes will be in duplex pending state until background copy completes GDPS / Hyperswap failover fails for volumes in Duplex Pending Resolve this issue by shipping function rather than data through PPRC (DS8K FlashCopy Preserve Mirror solution) No XRC solution in the short term V10: RECOVER New BACKOUT YES option for point in time recovery True rollback, not run of generated SQL undo statements Requires COPY YES for indexes Not for media recovery Allow fast data restore from data set level FlashCopy New ZPARM REC_FASTREPLICATION parameter PREFERED (default) use FlashCopy if available REQUIRED use FlashCopy for recovery NONE use standard I/O for recovery VERIFYSET option to fail PIT recovery if entire set not included Base, LOB, XML, history not RI ENFORCE option to avoid CHKP/ACHKP when subset of set recovered Improved performance due to avoidance of set checking (RI, aux) HSM Enhancements in z/os 1.12 QUERY COPYPOOL command to display background copy percent-complete information Allow RESTORE to use Sytem-level backups without waiting for FlashCopy background copy to complete Use DS8K Fast Reverse Restore (i.e. FRR) feature New SMS Copy Pool setting to enable FRR Source volumes can t be in any other FlashCopy relationships LIST COPYPOOL command to display whether FRR was enabled and the recovery status of using FRR

39 Multiple Space Efficient Versions Storage Group Name: PAYBACK Type: Copy Pool Backup Copy Pool: CP1 Versions: 3 Storage Group Name: DATA Type: Pool Copy Pool Backup Name: PAYBACK } Version n+2 2% 10% FRBACKUP CP1 2% data change since version created data change between versions 12% data change since version created Version n+1 12% }13% data change between versions 25% data change since version created Version n 25% Dump Tape Total 'real' storage: 39% CP1 Version n Summary A fast and non-disruptive backup solution using FlashCopy and DB2 Backup System Utility Support Incremental FlashCopy Copy Pool backups can be used as the source for DB2 table/index recovery Automatically manage Copy Pool backups on tapes Restore System Utility can recover DB2 system from DASD or Tapes Recover Utility can recover table space and index to PIT with transaction level consistency Remote Pair FlashCopy to alleviate PPRC restriction Tight collaboration between DB2 and Storage teams HSM Enhancements in z/os 1.12 Allow Backup System to use Space Efficient FlashCopy to keep backups on tapes CopyPool with Version = 0 Space Efficient Volumes Not consume physical space until writes take place Requires FlashCopy SE licensed feature HSM selects SE target volumes when Copy Pool is defined with NOCOPY (VERSIONS = 0) If FRR is enabled, recovery of the entire Copy Pool can be performed from the disk backup (in addition to a dump tape version) Questions

40

41 DB2 10 for z/os Virtual and Real Storage Management John Campbell Distinguished Engineer DB2 for z/os Development DB2 10: Massively Scalable DBMS Today DB2 10 Coupling Technology Coupling Technology LPAR1 LPAR2 LPAR3 LPAR1 LPAR2 LPAR3 DB2A (500 thds) DB2B (500 thds) DB2C (500 thds) DB2A (2500 thds) DB2B (2500 thds) DB2C (2500 thds) DB2D (500 thds) DB2E (500 thds) DB2F (500 thds) Data sharing and sysplex allows for efficient scale-out of DB2 images Sometimes multiple DB2s per LPAR More threads per DB2 image More efficient use of large n-ways Easier growth, lower costs, easier management Data sharing and Parallel Sysplex still required for very high availability and scale Rule of thumb: save ½% CPU for each member reduced, more on memory Disclaimer Copyright IBM Corporation All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE IBM, the IBM logo, ibm.com, and DB2 are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at 2 How Can You Maximize DB2 10 s Scalability? Terminology Virtual Storage Constraints and Management Real Storage Real storage consumption Real storage control system configuration parameters (ZPARMs) Understanding the key metrics via IFCID225 Future Considerations

42 Storage Management Terminology 64-Bit private storage 64-bit End of Virtual Above the Bar Extended Region 32-bit Bar Below the Bar Nucleus, ECSA, ESQA, etc. Nucleus, CSA, SQA, etc. Below the line 16 MB Line Virtual Storage Management Internal monitor will automatically issue console messages (DSNV508I) when storage crosses (increasing or decreasing) thresholds DSNVMON ) DB2 BELOW-THE-BAR STORAGE WARNING 77% CONSUMED 76% CONSUMED BY DB2 352M AVAILABLE OUT OF REGION SIZE 1553M WITH A 274M STORAGE CUSHION Threshold recently augmented in PM38435 to account for the storage cushion. Can get status at any time using DIS THREAD SERVICE(WAIT) NAME ST A REQ ID AUTHID PLAN ASID TOKEN VA1A N * VMON 01 SYSOPR 002A 0 V507-ACTIVE MONITOR, INTERVALS=8216, STG=77%, BOOSTS=0, HEALTH=100 REGION=1553M, AVAIL=352M, CUSHION=274M Good news DB2 10 have very little concern with virtual storage constraints after a REBIND! DB2 Scalability: Virtual Storage Constraints Thread 4 Thread 3 All Storage Extended Except Region the Storage Below Cushion The is Bar Consumed! Storage Thread 2 Thread 1 Storage Contraction Kicks Off Potentially Degrading Performance Only must complete processing succeeds and others get ABEND04E RC00E20**** DB2 Scalability: Real Storage Consumption 1000 s of threads Above the Bar Seemingly Without Bounds 16 exabytes 1000 threads 1000 threads 1000 threads 1000 threads ~1.5G

43 DB2 Scalability: Real Storage Consumption Above the Bar 16 exabytes Virtual Object 128G Real and Aux x Gigabytes Once Performance all aux is consumed, Degrades as the Real LPAR Storage goes into is Consumed a wait state Real Storage Control: Nothing to Fear! If we approach the REALSTORAGE_MAX threshold, we will automatically enter contraction mode to help pp protect the system Contraction mode start is indicated by a DSNV516I message DSNVMON BEGINNING STORAGE CONTRACTION MODE and ended by a DSNV517I message DSNVMON ENDING STORAGE CONTRACTION MODE When in contraction mode DB2 will attempt to discard all unused cached virtual pages. This unbacks virtual pages such that a real frame or aux slot is not consumed for this page. Actions include: More frequent agent local and shared system pool contraction Stack segments are contracted prior to going back on free queues Real Storage Control: Nothing to Fear! New system configuration parameters with apar PM24723 for DB2 10 to define real storage boundaries and how aggressively DB2 should strive to remain in the defined boundary. (Note: z/os apar OA35885 is required to enable the full functionality of PM24723) REALSTORAGE MAX defines the sandbox REALSTORAGE_MAX defines the sandbox Amount of real and aux in GB a given DB2 subsystem is allowed to consume If this value is hit, DB2 will terminate. The trade off is potential LPAR loss (Note: default is 0 which means NO LIMIT) If this value is approached a sticky DSNS003I is written to the console DSNVMON SUBSYSTEM WARNING FOR REAL STORAGE HAS BEEN DETECTED Upon relief a DSNS004I message is written and outstanding DSNS003I messages are DOM ed Real Storage Control: Nothing to Fear! The frequency of contraction mode can also be controlled by system parameter REALSTORAGE_MANAGEMENT. REALSTORAGE_MANAGEMENT defines the behaviour within the sandbox. REALSTORAGE_MANAGEMENT options include: OFF Do not enter contraction mode unless the REALSTORAGE_MAX boundary is approached OR z/os has notified us that there is a critical aux shortage ON Always operate in contraction mode. This may be desirable for LPAR with many DB2s or dev/test systems AUTO (the default) When significant paging is detected, contraction mode will be entered Important notes: Contraction mode is not exited immediately upon relief to avoid constant toggling in and out of this mode Contraction mode shows <1% CPU degradation

44 Illuminating Scalability: IFCID 225 IFCID 225 statistics records are critical for storage monitoring IFCID 225 Storage Summary Record SMF Record Type V8 = Type 102 V9/V10 = Type 100 Subtype 4 Enabled via STATISTICS trace CLASS 1 Interval if enabled V8/V9 = written at interval defined by ZPARM STATIME (recommendation is 1 minute) V10 = written at fixed 1 minute intervals Illuminating Scalability: IFCID Bit Common Shared Storage DBM1 Private DIST Private Below the Bar Real/Aux Real/Aux Shared 64-bit for 64-bit Common stack bufferpools thread Storage 64-bit for and DBM1 DIST Real/Aux for system DBM1 and DIST = = QW0225HVPagesInReal/QW0225AuxSlots associated excluding = Real/Aux QW0225RL/QW0225AX bufferpools objects = = = QW0225HVPagesInReal QW0225PriStg_Real/QW0225PriStg_Aux QW0225ShrStkStg_Real/QW0225ShrStkStg_Aux QW0225ShrStg_Real/QW0225ShrStg_Aux QW0225ComStg_Real/QW0225ComStg_Aux - Illuminating Scalability: IFCID 225 IFCID 225 changed dramatically in DB2 10 to account for the changes in storage usage The majority of SQL processing storage is in the DB2 shared object Data for the DIST address space is provided in addition to DBM1 64-bit common is introduced but lightly used 31-bit private monitoring is less important in particular if REBINDs are done 31 common (ECSA) storage counters are included 31-common (ECSA) storage counters are included Detailed real and aux storage counters are introduced in PM24723 to help with system monitoring and the setting of real storage control system configuration parameters OMPE apar PM40133 formats all new fields and presents totals Illuminating Scalability: IFCID 225 Real Storage Summary from OMPE (see reference fields at end for record details): REAL STORAGE IN USE SUMMARY Field (V10 only) /64-BIT PRIVATE (DBM1) (MB) /64-BIT PRIVATE (DIST) (MB) BIT SHARED THREAD AND SYSTEM (MB) BIT SHARED STACK (MB) BIT COMMON (MB) 10 TOTAL REAL STORAGE IN USE (MB) 4220

45 Illuminating Scalability: Storage Influences and Tuning Opportunities REAL STORAGE IN USE SUMMARY Field (V10 only) /64-BIT PRIVATE (DBM1) (MB) /64-BIT PRIVATE (DIST) (MB) BIT SHARED THREAD AND SYSTEM (MB) BIT SHARED STACK (MB) BIT COMMON (MB) 10 TOTAL REAL STORAGE IN USE (MB) 4220 This field includes buffer pools. Most likely this is the major influence. To be sure you can drill down with these fields: 64-BIT REAL 4K FRAMES IN USE : ABOVE VALUE W/O BP STORAGE : Considerations: buffer pool size, page fixed buffer pools (1M page sizes), possibly CONTSTOR and MINSTOR, REALSTORAGE_MANAGEMENT Illuminating Scalability: Storage Influences and Tuning Opportunities (1M Page Sizes) Issued by SRM indicating the system is using >=80% of the available large frames. The case if the number of large frames configured is close to the BP size. IRA120E LARGE FRAME SHORTAGE Display LFAREA configuration details for the LPAR IAR019I DISPLAY VIRTSTOR 984 SOURCE = AO TOTAL LFAREA = 200M LFAREA AVAILABLE = 126M LFAREA ALLOCATED (1M) = 74M LFAREA ALLOCATED (4K) = 0M << No broken down 1M frames MAX LFAREA ALLOCATED (1M) = 96M MAX LFAREA ALLOCATED (4K) = 0M Serviceability command to display buffer pool allocations -DIS BUFFERPOOL SERVICE(4) DSNB999I =D2V1 DSNB1DBP SERVICE( 4 )OUTPUT DSNB999I =D2V1 4K PAGES 0 DSNB999I =D2V1 1M PAGES 1476 Illuminating Scalability: Storage Influences and Tuning Opportunities (1M Page Sizes) Before v10, DB2 used only 4K page sizes. Now with z10 s and DB2 v10, DB2 exploits the performance benefits of 1M page sizes. Reduction of misses in the TLB (translation lookaside buffer) Up to 4% CPU reduction observed in beta customer and lab testing LFAREA in IEASYSxx defines how storage can be used for 1M page sizes at IPL. These pages are fixed and will not be paged to AUX. 1M pages can be broken down and given to short term fix swappable address spaces (this would not be a DB2) If only DB2 on the system, it is best to get the setting of LFAREA close to the amount that will be used else paging may occur due to lack of 4K page availability. 1M page sizes are available for page fixed buffer pools Enabled if PGFIX(YES) Enabled if PGFIX(YES) Attempt to get 1M pages if available, else will get 4K pages A buffer pool can be a mix of 1M and 4K pages depending on availability Once allocated, pages will not be reconfigured even if 1M pages later become available Illuminating Scalability: Storage Influences and Tuning Opportunities REAL STORAGE IN USE SUMMARY Field (V10 only) /64-BIT PRIVATE (DBM1) (MB) /64-BIT PRIVATE (DIST) (MB) BIT SHARED THREAD AND SYSTEM (MB) BIT SHARED STACK (MB) BIT COMMON (MB) 10 TOTAL REAL STORAGE IN USE (MB) 4220 This is the majority of thread storage needed to run SQL. This can be compared against the number of agents in the system as OMPE will report active threads, system agents, DBATs, and parallel child threads. Considerations: MAXDBAT, CTHREAD, PARAMDEG, SET CURRENT DEGREE or DEGREE bind options, REALSTORAGE_MANAGEMENT

46 Illuminating Scalability: Storage Influences and Tuning Opportunities REAL STORAGE IN USE SUMMARY Field (V10 only) /64-BIT PRIVATE (DBM1) (MB) /64-BIT PRIVATE (DIST) (MB) BIT SHARED THREAD AND SYSTEM (MB) BIT SHARED STACK (MB) BIT COMMON (MB) 10 TOTAL REAL STORAGE IN USE (MB) 4220 This is the majority of stack required by threads to execute SQL. This can be compared against the number of agents in the system as OMPE will report active threads, system agents, DBATs, and parallel child threads. Considerations: MAXDBAT, CTHREAD, PARAMDEG, SET CURRENT DEGREE or DEGREE bind options, REALSTORAGE_MANAGEMENT Illuminating Scalability: Storage Influences and Tuning Opportunities 31-bit common (ECSA) may become a scaling factor since many DB2 agent structures still remain in ECSA EXTENDED CSA SIZE : BIT COMMON VARIABLE POOL : BIT COMMON FIXED POOL : BIT COMMON GETMAINED : Considerations: Since 31-bit storage is far less constrained, it may be possible to increase the amount of ECSA defined if multiple subsystems stems are being consolidated CTHREAD, MAXDBAT, PARAMDEG, SET CURRENT DEGREE or DEGREE bind options Number of DB2s on a given LPAR Illuminating Scalability: Storage Influences and Tuning Opportunities REAL STORAGE IN USE SUMMARY Field (V10 only) /64-BIT PRIVATE (DBM1) (MB) /64-BIT PRIVATE (DIST) (MB) BIT SHARED THREAD AND SYSTEM (MB) BIT SHARED STACK (MB) BIT COMMON (MB) 10 TOTAL REAL STORAGE IN USE (MB) 4220 This is lightly used in DB2 10. The majority of this storage is from distributed agent structures, t package accounting, and rollup accounting. Considerations: MAXDBAT, PARAMDEG, SET CURRENT DEGREE or DEGREE bind options, whether ACCTG/MON classes 7 or 8 are enabled (package accounting), ACCUMACC, and possibly CTHREAD and REALSTORAGE_MANAGEMENT Illuminating Scalability: Further Tuning It is recommended that REALSTORAGE_MAX be set to a multiple of the maximum storage a system is expected to consume. This may require monitoring over a period of time. MEMU2 and MEMUSAGE can help display trends over time. These have moved to the DB2 for z/os Exchange community website on IBM My developerworks. (1) From IBM My developerworks My Home (sign in with your IBM login at search 'memu2' in All My developerworks. (2) From DB2 for z/os Exchange ( click on 'View and download examples'. The file is tagged with 'memu2'. (3) To access MEMU2 directly: These trends can also help with a proper setting for MAXSPACE. In v8/v9 the recommendation is 8G and in v10 this will likely be larger (10G 12G). Also consider how many DB2s may be on a given LPAR when setting this parameter.

47 Scalability: Future Considerations Link system health to real-storage and other constraints. This is reported to WLM and some work can be directed around constrained systems (see DIS THD(*) TYPE(SYSTEM) message DSNV507I messages for system health reporting). Reduce query parallelism degree in the face of real-storage or other constraints We will be seeking to reduce our 31-bit ECSA footprint in future releases of DB2 Additional 31-bit storage will be moved above the bar in future releases Questions

48

49 SQLADRIA SEMINAR Ljubljana, 13 April 2012 Loading a Data Warehouse Zdenka Gazvoda, Jure Lovšin ZZZS Sponsored by