Session: B10 Should Your Next Warehouse be on DB2 for z/os? Willie Favero IBM Silicon Valley Lab. 15 October :30 14:30 Platform: z/os

Transcription

1 Session: B10 Should Your Next Warehouse be on DB2 for z/os? Willie Favero IBM Silicon Valley Lab 15 October :30 14:30 Platform: z/os

2 Disclaimer The information contained in this presentation has not been submitted to any formal IBM review and is distributed on an "As Is" basis without any warranty either expressed or implied. The use of this information is a customer responsibility. The materials in this presentation are also subject to enhancements at some future date a new release of DB2, or a Programming Temporary Fix (PTF) IBM MAY HAVE PATENTS OR PENDING PATENT APPLICATIONS COVERING SUBJECT MATTER IN THIS DOCUMENT. THE FURNISHING OF THIS DOCUMENT DOES NOT IMPLY GIVING LICENSE TO THESE PATENTS. 2

3 Trademarks AIX* CICS* DB2* DB2 Connect DB2 Universal Database Distributed Relational Database Architecture DRDA* ESCON* FICON* GDPS* HiperSockets * Registered trademarks of IBM Corporation IBM* IBM eserver IBM logo* IMS OMEGAMON* On Demand Business logo Parallel Sysplex* QMF RMF System z System z9 Tivoli* WebSphere* z/architecture z/os* z/vm* zseries* The following are trademarks or registered trademarks of other companies. Intel is a trademark of the Intel Corporation in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Java and all Java-related trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States and other countries. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. UNIX is a registered trademark of The Open Group in the United States and other countries. * All other products may be trademarks or registered trademarks of their respective companies. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography. This presentation and the claims outlined in it were reviewed for compliance with US law. Adaptations of these claims for use in other geographies must be reviewed by the local country counsel for compliance with local laws. For additional information visit the URL for Copyright and trademark information 3

4 You re right Way too many foils!!!! There is so much good stuff, I just couldn t decide what to leave out. BTW, the intent of this presentation is not to cover every slide, but rather provide detailed information on how data warehousing on System z uses DB2 for z/os

5 Terminology What s been in DB2 to Support DW DB2 V8 DW Contributions DB2 9 DW Contributions Some Hardware Stuff 5

6 Agenda Terminology What s been in DB2 to Support DW DB2 V8 DW Contributions DB2 9 DW Contributions Some Hardware Stuff 6

7 Terminology Data warehouse From Webster s a computer system that collects, stores, and manages large amounts of data for a company, to be used for business analysis and strategy 7

8 Terminology Data mart From Webster s a type of data warehouse designed primarily to address a specific function or department's needs, as opposed to a data warehouse which is traditionally meant to address the needs of the organization from an enterprise perspective. In addition, a data mart often uses aggregation or summarization of the data to enhance query performance. 8

9 Terminology Business Intelligence (BI) and Data Warehousing (DWH) are sometimes used interchangeably BI includes end user tools for query, reporting, analysis, dashboarding etc. Both concepts depend on each other BI almost always assumes a Data Warehouse (DWH), Operational Data Store (ODS) or Data Mart (DM) exists with timely, trusted information An ODS is a subject oriented database organized by business area. up to date (vs. historical) detailed (vs. summarized). A DM is a database designed to support the analysis of a particular business subject area. data has usually been transformed and aggregated from the source DWH or operational system could be relational, multidimensional (OLAP) or statistical in nature. A DWH depends on end user tools that turn data into information. Both terms (DWH and BI) address desire for timely, accurate, available data delivered when, where and how the end users want it. 9

10 Terminology Operational data store (ODS) From Bill Inmom an ODS is an integrated, subject- oriented, volatile (including update), current-valued structure designed to serve operational users as they do high performance integrated processing 10

11 Terminology On-Line Analytical Processing (OLAP) From Webster s a category of database software which provides an interface such that users can transform or limit raw data according to user-defined or pre-defined functions, and quickly and interactively examine the results in various dimensions of the data 11

12 Need a Good Decision Support System? "DB2 offers a powerful new data base management system for use where business and application requirements and data structures are subject to frequent changes. Its ease-of-use characteristics make it applicable to those applications in which rapid implementation are the primary requirement. As such, DB2 may be used to address decision support systems as well as traditional application areas." June 7, 1983 IBM announcement letter titled Relational Data Base Management System, End-User Query, And Data Extract 12

13 But What Really Started It All Published in the IBM System Journal, Vol 27, No 1, 1988 An Architecture for a Business and Information System by B. A. Devlin, P. T. Murphy 13

14 Traditional Topology Operational systems Extract, transform, load Operational data store ETL metadata Enterprise Data Warehouse metadata Line of Business Data Marts Dependent Data Marts Independent Data Mart 14

15 DB2 for z/os: Back into the ring July 2006 Announcement NUS /ENUS PDF Announces availability of products to support the mission The time is right, shift in customer requirements Real-Time data access Service Level Agreements to match Operational Operational BI and Embedded Analytics Deliver BI to customer facing humans and applications (broad audience) Integrate BI Components with Operational Systems & Information Portals Massive number of queries Queries semi-aggregated data, i.e. data aggregated at a low level Executive Manager Analyst Customer Service Customers < 50 > 10,

16 A New Approach to Leveraging Information Information On Demand to Optimize Real-Time Processes Dynamic Warehousing OLAP & Data Mining to Understand Why and Recommend Future Action Traditional Data Warehousing Query & Reporting to Understand What Happened 16

17 So what does DB2 for z/os have to offer the data warehouse world? Again! 17

18 Terminology What s been in DB2 to Support DW DB2 V8 DW Contributions DB2 9 DW Contributions Some Hardware Stuff 18

19 First, a quick look at what has been there for quite some time 19

20 The #1 Optimizer DB2 V1.1 The best optimizer in the business since its beginning Cost based using Catalog statistics Database design SQL statement details Central processor model Number of central processors Used to determine appropriate degrees of parallelism Buffer pool size RID pool size Constantly being improved With new versions With PTFs between version releases And to insure you are getting the best access path possible You can use Optimization Service Center (OSC) to tune queries and query workloads 20

21 Resource Limit Facility (RLF) V2.1 Enables limitation of resource usage bind operations parallelism modes in dynamic query processing Used with dynamic SQL statements (SELECT, INSERT, UPDATE, MERGE, TRUNCATE, or DELETE ) only RLF has two mode Reactive mode Stop dynamic SQL statements that exceed specified processor limits Predictive mode Set warning and error thresholds for dynamic SQL statements that a processing limit might be exceeded Restrict bind and rebind activities to avoid performance impacts on production data Important for a data warehousing application 21

22 Data Compression V3 Compression should always be considered for a data warehouse Savings are usually greater than 50% Have seen as high as 80% in certain situations Overhead on INSERT minimal on SELECT Warehouse queries dominated by sequential prefetch, which benefit from DB2 compression. Not all rows in a table spaces can be compressed If the row after compression is not shorter than the original uncompressed row, the row remains uncompressed. Compression dictionary size 64K (16 X 4K page) of storage in the DBM1 address space Dictionary goes above the bar in DB2 Version 8 and later releases Faster hardware, faster compression See hardware chart toward end of presentation 22

23 Dictionary 4K Page Size Data pages (18-???) Dictionary pages (20-17) The dictionary is created by the LOAD and/or REORG utilities only It occupies: 4K 16 pages 8K - 8 pages 16K - 4 pages 32K - 2 pages The compression dictionary follows the header and first space map pages (next slide) Dictionaries can be at the partition level (Careful, you could have 4096 partitions Space map page (1) Header page (0) 23

24 Data Compression Rows are compressed on INSERT For an UPDATE Expand, update, then re-compressed row UPDATE has the potential to be expensive Changes (INSERT & UPDATE) are logged in compressed format Possible reduced logging cost Active log reductions carried over to the archive logs Larger page sizes may result in better compression. Resulting rows after compression are variable length You might be able to fit more rows with less wasted space in a larger page size. You cannot turn compression on for the catalog, directory, work files, or LOB table spaces Index compression does not use a dictionary 24

25 Possible Performance Gain When compression is on, data pages are brought into buffer pool in compressed state Increasing the number of rows in the same size pool could increase buffer pool hit ratio Increasing hit ratio could reduce I/O necessary to satisfy the same number of getpage requests. If Compression doubles the number of rows per page When DB2 loads that page in a buffer pool, it will be loading twice as many rows. Less I/O is always a good thing. 25

26 Determining When to Use Compression DSN1COMP Stand-alone utility Will estimate disk savings Works for table spaces and indexes Can be run against: Table space underlying VSAM dataset Index space underlying VSAM dataset Output from a full image copy Output from DSNCOPY Cannot be run against: LOB table spaces Catalog (DSNDB06) Directory (DSNDB01) Workfiles (i.e. DSNDB07) Using DSN1COMP with image copies and DSN1COPY outputs can make gathering compression information unobtrusive 26

27 Parallelism in DB2 V3, V4, and V5 27

28 I/O Parallelism SELECT * FROM TAB1 A, TAB2 B WHERE A.COLA=B.COLZ ORDER BY A.COLM Introduced in DB2 V3 DB2A DB2B DB2C S y s p l e x ziip eligible

29 CP Parallelism SELECT * FROM TAB1 A, TAB2 B WHERE A.COLA=B.COLZ ORDER BY A.COLM Introduced in DB2 V4.1 DB2A DB2B DB2C S y s p l e x 1 2 ziip eligible 3 29

30 Sysplex Query Parallelism SELECT * FROM TAB1 A, TAB2 B WHERE A.COLA=B.COLZ ORDER BY A.COLM Introduced in DB2 V5 DB2A Data Sharing Group DB2B DB2C S y s p l e x ziip eligible

31 Set Parallelism s Max. Degree Set the maximum degree between the # of CPs and the # of partitions CPU intensive queries - closer to the # of CPs I/O intensive queries - closer to the # of partitions Data skew can reduce # of degrees 31

32 Star Schema Dimension Table Dimension Table Fact Table Dimension Table Dimension Table Dimension Table Star schema = Snowflake schema 32

33 Star Schema Store Product Sale Date Three or more tables can be joined at one time ZPARMs SJTABLES and STARJOIN must be set ziip eligible Promotion Associate Star schema = Snowflake schema 33

34 AL CT DE FL IA IL IN KY LA MA MD MI MN MO MS NC NH NJ NY 106, JAN, KY 101, FEB, DE 102, MAR,MO 105, APR, CT 104, NOV, IL 103, DEC, MI 205, JAN, AL 206, FEB, NC 201, JUL, NJ 202, AUG, FL 203, OCT, IA 204, DEC, MD 304, APR, MS 303, MAY,MN 306, JUN, NH 302, JUL, MD 305, SEP, CT 301, NOV, LA 404, FEB, IN 403, MAR, FL 405, JUN, MA 401, SEP, NY 406, OCT, NH 402, NOV, IA JAN FEB MAR APR NOV DEC JAN FEB JUL AUG OCT DEC APR MAY JUN JUL SEP NOV FEB MAR JUN SEP OCT NOV DB2 for z/os Partitioning Partitioning in DB2 for z/os V8 and beyond is defined at the table level Maximum of 4096 partitions DB2 can generate a maximum of 254 parallel operations Effectively cluster by two dimensions Partition by Growth DB2 9 feature that relates (in a way) to hash Secondary Index -- partitioned like the underlying data (DPSI) Partitioned table Secondary Index -- non-partitioned (NPSI) 34 One example

35 Data Sharing DB2 V5 Member A OLTP Member CEC One C DWH CF Member B OLTP Member CEC Two D DWH DB2 Data Sharing DB2 Group Data Sharing Group CEC One CF CEC Two Within a data sharing environment, the data warehouse can reside in the same group as the transactional data. 35

36 Noteworthy DSNZPARMs for Warehousing Recommendation for DWH CDSSRDEF=ANY, CONTSTOR=NO, DSVCI=YES, MGEXTSZ=YES, OPTCCOS4=ON, Comments ANY Allow parallelism for DW. any: parallelism, 1: no parallelism NO For best performance, specify NO for this parameter. To resolve storage constraints in DBM1 address space, specify YES. See also: MINSTOR YES The DB2-managed data set has a VSAM control interval that corresponds to the buffer pool that is used for the table space. YES Secondary extent allocations for DB2-managed data sets are to be sized according to a sliding scale enables fix PK26760 (inefficient access plan) OPTIXIO=ON OPTIORC=ON OPTIXIO=ON: Provides stable I/O costing with significantly less sensitivity to buffer pool sizes. (This is the new default and recommended setting). OPTIORC=ON explanation??? PARAMDEG=X, #Processors <= X <= 2*#Processors If concurrency level is low, the ratio can be higher. SRTPOOL=8000, 8000 (means 8 MB Sort Pool) STARJOIN=DISABLE, MXQDC=15 DISABLE, unless SAP BW on z/os used Default is 15. If changed, set MXQDC=TABLES_JOINED_T HRESHOLD*(2**N)-1 36

37 Terminology What s been in DB2 to Support DW DB2 V8 DW Contributions DB2 9 DW Contributions Some Hardware Stuff 37

38 Business Intelligence & DB2 V8 Performance Data-partitioned secondary indexes (DPSI) Multiple DISTINCT clauses in SQL statements Reduced lock contention on volatile tables Coupling Facility lock propagation reduction Multi-row INSERT/FETCH REOPT(ONCE) to reduce host variables impact on access paths Index-only access for VARCHAR columns Backward index scan Faster short PREPARE IN access path performance DDF performance enhancements Business warehouse Sparse index for star join More tables in join Common table expressions Recursive SQL Materialized query tables Continuous availability Changing clustering index as online operation Elimination of BUILD2 phase of REORG with DPSIs Online schema evolution for many column types Volume-level, automated backup and recovery CI size larger than 4 KB More log data sets Conditional restart enhancements Support for synchronizing log point Architecture Unicode support Introduction of DB2 Connect DB2 Universal Driver for JDBC 64-bit virtual storage for most DB2 storage areas Up to 4096 partitions Longer table/column names SQL statements up to 2 MB ASCII precompiler 50+ new features Ease of use Clustering decoupled from partitioning New REORG option to reorganize all partitions in Reorg-pending state CREATE INDEX invalidates statements from dynamic statement cache Indexes created as deferred are ignored by DB2 optimizer LOB ROWID transparency Collecting distribution statistics on arbitrary sets of columns with RUNSTATS Fast cached SQL invalidation Automatic space management Statements IDs of cached statements as input to EXPLAIN Statement ID in IFCID 124 Long-running non-committing reader alerts Lock escalation reporting Transaction-based DB2 accounting and workload management Stored procedures to facilitate database administration Network statistics with DB2 Connect DRDA ping Comments in dynamic SQL CTE-based optimizer hints 38

39 DB2 V8 Enhancements Benefiting DW Usability, Availability & Scalability Online Partitioning Changes, Rotate Schema Evolution: ALTER instead of DROP & CREATE VSCR with System z 64-bit Architecture Overall System & Query Performance Locking Improvements Index Access for VARCHAR, data type mismatch, DDF Performance Improvements Data Warehousing Support Star Join Improvements Optimization Service Center available for Version 8 39

40 Terminology What s been in DB2 to Support DW DB2 V8 DW Contributions DB2 9 DW Contributions Some Hardware Stuff 40

41 Business Intelligence and DB2 9: Performance New row internal structure for faster VARCHAR processing Fast delete of all the rows in a partition (TRUNCATE) Numerous enhancements in 'smaller' LOB performance Fast LOB streaming Reducing log latch contention Deleting first n rows Skipping uncommitted inserted/updated qualifying rows Faster release of LOB locks Reducing data sharing overhead for global indexes Index on Expression Business warehouse Dynamic index ANDing Reduce temporary tables materialization Generalizing sparse index/inmemory data caching Continuous Availability Universal Table Spaces Partition-by-growth as a means to remove non-partitioned tablespace size limit Full support for system-level backup and recover (automatic offload to tapes and individual objects recovery) Renaming SCHEMA and VCAT to facilitate fast database provisioning Rename index Reorganization of LOBs to reclaim space Online REORG enhancements Online REBUILD index Architecture/SQL Thin DB2 Connect Client FOR BIT DATA collating sequence (VARBINARY) Full JDBC compliance Enable Decimal Float data type (preconditioning) BIGINT data type Index compression Architecture/SQL (con t) Provide more VS relief for thread related storage (partially) Unicode support for all CLI functions MERGE statement SET operations Ease of Use Implicit objects creation Enhancing real time statistics (Optimization Service Center) Autonomic reoptimization Integration of Real Time Statistics tables into the catalog Simulating indexes in EXPLAIN (Optimization Service Center) More autonomic bufferpools tuning (WLM synergy) RLF support for end-user correlation TRACE support for end-user correlation Enhance tracing in DB2 Connect Identifying unused indexes Enhancing IFC for IRLM diagnostics DSNACCOR enhancements 41

42 DB2 9 enhancements benefiting DW Usability, Availability & Scalability Clone Tables (Online Load Replace) XML, text, and spatial data Overall System & Query Performance Significant CPU Reduction for DB2 utilities & many queries Native SQL stored procedures Data Warehousing Support: enhanced data, techniques, OSC & OE Star Join Improvements (Dynamic Index ANDing) SQL Enhancements (RANK, DENSE_RANK, INTERSECT, Except, ) Optimization Service Center 42

43 DB2 Index Compression.. Index compression is new to V9 Page level compression Unlike data row compression: Buffers contain expanded pages Pages are decompressed when read from dasd Prefetch performs the decompression asynchronously A buffer hit does not need to decompress Pages are compressed by the deferred write engine Like data row compression: An I/O bound scan will run faster DSN1COMP utility can be used to predict space savings 43

44 Index Compression Example Suppose an index could compress 3-to-1 Decompressed 16K buffer Compressed 4K Decompressed 8K buffer Compressed 4K 12K used 4K unused 8K Buffer pool 50% disk space reduction No increase in virtual storage cost 16K Buffer pool 67% disk space reduction 33% increase in virtual storage cost 44

45 ..DB2 Index Compression The CI Size of a compressed index on disk is always 4K A 4K expands into a 8K or 16K buffer, which is the DBA s choice. This choice determines the maximum compression ratio. Compression of key prefix and Rid Lists A Rid List describes all of the rows for a particular index key An index with a high level of non-uniqueness, producing long Rid Lists, achieves about 1.4-to-1 compression Compression of unique keys depends on prefix commonality 45

46 Terminology What s been in DB2 to Support DW DB2 V8 DW Contributions DB2 9 DW Contributions Some Hardware Stuff 46

47 Of course, there is always the operating system and hardware! 47

48 The Mainframe is Dead?????? 48 MIPS Shipped

49 Where s the Data? 25 of the top 25 world-wide banks run on DB2 for z/os 23 of the top 25 US retailers run on DB2 for z/os 9 of the top 10 global life or health insurance providers run on DB2 for z/os 95% of US Fortune 500 companies are System z clients. 45% of US Fortune 1000 companies are System z clients. 71% of global Fortune 500 companies are System z clients. 80% of world s corporate data resides or originates on a mainframe 49

50 And It Just Keeps Getting Faster The faster the chip speed, the better compression performs z GHz New.58 z9 1.7 GHz Chip Speeds (nanoseconds) G6 (9672) 550 MHz z800 z MHz z GHz 1.8 Compression introduction On 9672 or a 711- or 511-based ES/9000 Chart NOT to any kind of scale New Processors Introduced 50

51 Improved Server Performance and scalability with faster and more processors and improved dispatching synergy The z10 EC delivers on average 50% more performance in a n-way configuration The uniprocessor is expected to deliver 62% more performance than z9 EC uniprocessor * The z10 EC 64-way offers 70% more server capacity than the largest z9 EC** Introducing HiperDispatch for improved synergy with z/os operating system to deliver scalability and performance Customer Engines IFL Crypto z900 z990 zaap z9 EC ziip z10 EC 4.4 GHz processor chip Hardware Decimal Floating Point Capacity * LSPR mixed workload average running z/os z10 EC 701 versus z9 EC 701 ** This is a comparison of the z10 EC 64-way and the z9 EC S54 and is based on LSPR mixed workload average running z/os 1.8 Significant capacity for traditional growth and consolidation 51

52 System z9 Integrated Information Processor The latest specialty engine for the System z9 mainframe designed to help: Customers integrate data across the enterprise Improve resource optimization and lower the cost of ownership for eligible data serving workloads z/os manages and directs work between the general purpose processor and the ziip Number of ziips per IBM System z9 not to exceed number of standard processors No changes anticipated to DB2 UDB for z/os V8 applications No IBM software charges on the ziip consistent with other specialty engines DB2 for z/os V8 will be first IBM exploiter of ziip System z9 EC, z9 BC, or z10 EC z/os 1.6 or later (with PTFs) DB2 UDB for z/os V8 (with PTFs) 52

53 ziip Eligible Workloads Notes Portions of the following DB2 UDB for z/os V8 workloads may benefit from ziip* ERP, CRM, Business Intelligence or other enterprise applications For applications, running on z/os, UNIX, Linux, Intel, or Linux on System z that access DB2 for z/os V8 (and above) on a System z9, via DRDA over a TCP/IP connection. DB2 gives z/os the necessary information to have portions of these SQL requests directed to the ziip Data warehousing applications* Requests that utilize complex large parallel queries (including star schema queries) may have portions of these SQL requests directed to the ziip when DB2 gives z/os the necessary information DB2 UDB for z/os V8 utilities* Requests that utilize DB2 UDB for z/os V8 complex star schema parallel queries may have portions of these SQL requests directed to the ziip when DB2 gives z/os the necessary information Utilization of the ziip is expected to be transparent to the application. No anticipated changes to applications that access DB2 UDB for z/os V8 data 53

54 Enterprise Applications Enterprise Applications that access DB2 UDB for z/os V8 via DRDA over a TCP/IP connection will have portions of these SQL requests directed to the ziip Notes CP High utilization CP ziip DB2/DRDA DB2/DRDA Ent App TCP/IP (via Network or HiperSockets) DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA Portions of eligible DB2 enclave SRB workload executed on ziip DB2/DRDA DB2/DRDA DB2/DRDA DB2/Batch DB2/Batch DB2/DRDA For illustrative purposes only. Single application only. Actual workload redirects may vary Reduced utilization 54

55 Focus on DRDA A portion of DB2 SQL processing coming in through DRDA via TCP/IP can be eligible for ziips One objective with the ziip is to help bring the costs of network access to DB2 more closely in line with the costs of running similar workloads under CICS, IMS, or Batch on the mainframe. Why focus on DRDA? Database workloads such as CICS, IMS, stored procedures and Batch have become increasingly efficient and cost effective on the mainframe. Today, customers are looking to further leverage their data on the mainframe, and are turning to the mainframe more and more frequently for new application workloads. These application workloads, such as Enterprise Resource Planning, Customer Relationship Management and Business Intelligence often use DB2 as a database server. Added system costs for network access to DB2 results in hardware and software charges that are substantially higher for remote applications, when compared to the same workload running as a local batch application. The ziip is aimed at helping customers more cost effectively run these database serving applications on the mainframe. Notes 55

56 What is DRDA? Notes DRDA = Distributed Relational Database Architecture Developed by IBM App or DB Enables relational data to be distributed among multiple platforms any app to any db and any db to any db DRDA is implemented in DB2 UDB for z/os V8 and reduces the need for additional gateway products that may affect performance and availability. The Open Group adopted DRDA in 1998 as the open standard for database access interoperability. An application uses DRDA application requestor or server to access a remote database. DB2 Connect is an example of a DRDA application server. The universal driver is an example of a DRDA application requester DRDA is network independent. It can use TCP/IP or SNA as a network protocol to flow DRDA commands. Connections using SNA are not eligible for ziip. So... regarding the ziip: If DB2 for z/os V8 work load comes over TCP/IP and is DRDA compliant (and not stored procedures or user-defined functions), a portion of that DB2 workload is eligible to be redirected to the ziip you need BOTH TCP/IP and DRDA but not stored procedures or user-defined functions. D R D A DB 56

57 What are enclave SRBs? z/os dispatches work in either TCB (Task Control Block) mode or SRB (Service Request Block) mode. DB2 parallel tasks use SRB mode and are assigned the same importance as the originating address space. Preemptible enclaves are used to do the work on behalf of the originating TCB or SRB address space. Enclaves are grouped by common characteristics and service requests and since they are preemptible, the z/os dispatcher (and WLM) can interrupt these tasks for more important ones (i.e. manage a transaction end-to-end). There are two types of preemptible SRBs: client SRBs and enclave SRBs. If the DB2 for z/os V8 request is coming in over distributed (i.e. DRDA over TCP/IP) then most of that work (other than stored procedures and userdefined functions) is executed in enclave SRBs. If the request is coming over local / native connection, then that work is dispatched between TCBs, client SRBs, and enclave SRBs (star schema parallel queries and some utility index maintenance now use enclave SRBs) As for the ziip, only the enclave SRB work (not the client SRB work or non-preemptible SRB work or TCB work) is eligible to be redirected to the ziip Notes DB2 V8 knows how its work is dispatched and directs z/os 1.6 or later to dispatch (redirect) a portion of the eligible work to the ziip 57

58 Business Intelligence Applications (DRDA) Complex parallel queries via DRDA over a TCP/IP connection will have portions of this work directed to the ziip CP High utilization CP Notes Add ziip BI App TCP/IP DB2/DRDA/StSch DB2/DRDA/StSch DB2/DRDA/StSch DB2/DRDA/StSch DB2/DRDA/StSch DB2/DRDA DB2/DRDA DB2/DRDA/StSch Portions of eligible DB2 enclave SRB workload executed on ziip (via Network or HiperSockets ) DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA DB2/DRDA Reduced utilization DB2/DRDA DB2/Batch DB2/Batch For illustrative purposes only. Single application only. Actual workload redirects may vary depending on how long the queries run, how much parallelism is used, and the number of ziips and CPs employed 58

59 BI Applications (local no DRDA) Long running parallel queries via LOCAL connection will have portions of this work directed to the ziip Notes Add ziip CP High utilization CP BI App LOCAL NO DB2 DRDA DB2/StSch DB2/StSch DB2/StSch DB2/StSch DB2/StSch DB2 DB2 DB2 DB2 DB2 DB2/StSch DB2 DB2 DB2 DB2 Reduced utilization DB2 Portions of eligible DB2 enclave SRB workload executed on ziip DB2/Batch DB2/Batch For illustrative purposes only. Single application only. Actual workload redirects may vary depending on how long the queries run and how much parallelism is used 59

60 What is Star Schema? Notes Star schema = a relational database schema for representing multidimensional data Sometimes graphically represented as a star Data stored in a central fact table Surrounded by additional dimension tables holding information on each perspective of the data Example: store "facts" of the sale (units sold, price,..) with product, time, customer, and store keys in a central fact table. Store full descriptive detail for each keys in surrounding dimension tables. This allows you to avoid redundantly storing this information (such as product description) for each individual transaction Complex star schema parallel queries include the acts of joining several dimensions of a star schema data set (like promotion vs. product). So... regarding ziip: if the workload uses DB2 UDB for z/os V8 to join star schemas, then portions of that DB2 workload will be eligible to be redirected to the ziip. 60

61 Focus on Long Running Parallel Queries Notes Long running parallel query workloads may benefit from two redirected tasks 1. Main task = the DRDA request If the request is coming in via DRDA via TCP/IP it can take advantage of the DRDA use of ziip, just like any other network attached Enterprise Application. 2. Child task = the long running parallel queries If the business intelligence and data warehousing application uses star schemas, then a significant amount of this task (star schema) processing is eligible to be redirected to the zip. The child (star schema) & main tasks (coming in through DRDA via TCP/IP) are additive. Combining the child and the main tasks is expected to yield a larger amount of redirect than that of just DRDA via TCP/IP alone. Longer running queries see higher benefit. Benefits to a data warehousing application may vary significantly depending on the details of that application. 61

62 Focus on DB2 for z/os utilities Only the portions of DB2 utility processing related to index maintenance are redirected. Only the BUILD portion of LOAD, REORG, and REBUILD Amount of workload eligible for ziip will depend on: How many indexes are defined on the table How many partitions are in the table If data compression is being used Possibly other factors Lower amount eligible is expected with: Tables with fewer indexes Fewer partitions Compression used Higher amount eligible is expected with: LOAD and REORG with many indexes or many partitions Notes 62

63 ziip DRDA workload estimation Notes Please see your IBM representative for ziip data collection guidelines Data collection will depend on the type of workload you have, for example: For DRDA workloads that do not use (or only minimally use) DB2 Stored Procedures and/ or user defined functions, then data from RMF Workload Activity report should be sufficient For DRDA workloads that use DB2 Stored Procedures and/ or user defined functions extensively, then data from RMF Workload Activity and DB2 Accounting data should be used Data collection will also depend on the level of detail required, for example: Single application workloads may only need one day s peak workload activity Multiple applications may need a week s worth of data 63

64 Measuring ziip activity Once a ziip is installed (with z/os R6 or R7 (w/ PTF) and DB2 V8 (w/ PTFs)), monitoring ziip activity will be similar to monitoring zaap activity Set up WLM policy with Service Class(es) for SUBSYSTEM TYPE=DDF RMF Monitor 1 Type 70 Record will monitor overall ziip activity: Logical processor busy as seen by z/os is reported Physical processor busy as seen by LPAR is reported RMF Monitor 1 Type 72 Record will show more detail: The amount of time spent executing on ziip processors is reported Usage and Delay sample counts for ziip eligible work is reported In addition, DB2 accounting trace records can provide information on the ziip. IBM Tivoli OMEGAMON XE for DB2 Performance Expert on z/os, DB2 Performance Expert or IBM Tivoli OMEGAMON XE for DB2 Performance Monitor on z/os can be used to monitor the ziip information. Notes ProjectCPU provides accurate estimates for potential to redirect work to the ziip Requires z/os v1.6 or higher and DB2 V8 plus ziip PTFs z/os IEAOPTxx parmlib member 64

65 ziip Support APARs (not all inclusive) PK DRDA PK Utilities, index management PK Utilities, index management PK Star schema PK Long running parallel queries PK Omegamon XE for DB2 PE (5655-P07) and Omegamon XE for DB2 PM (5655-P08) PK SDSF OA RMF monitoring support OA SMF service time reporting OA Omegamon II for MVS v550 (component of Omegamon XE for z/os even though it is a stand alone product) OA OMEGAMON Base v550 (component of the above product) OA OMEGAMON XE on z/os v310 FMID JBB77S9 - necessary ziip function support for z/os 1.6. FMID JBB772S - necessary ziip function support for z/os 1.7. II14219 ziip exploitation informational APAR Notes 65

66 Specialty Processors Member A OLTP Member B OLTP Extract Member C Member D ziip ETL Tools DWH Mgmt DWH DWH CEC One CEC Two Load Query BI Tools CP z/os ICF CP z/os IFL Linux on z ICF Uniquely allows a Data Warehouse database to coexist with an OLTP database IFL Enables efficient data movement (secure, high-speed hipersockets) Lowers TCO through reduced hardware and software costs Enables use of ziips during extract and further reduces costs ziip Further enables lower cost of Business Intelligence queries 66

67 A Product view OLTP data Information Server for System z Complete ETL Solution Data Warehouse DB2 for z/os The Enterprise Data Warehouse Cognos 8 BI for System z Core Offering for Enterprise Data Warehouse and BI: Information Server for System z A complete set of ETL tools for warehouse population and management DB2 for z/os, including the new Value Unit Edition A new value point for new DB2 for z/os workloads Coming Soon! Cognos 8 BI for System z (beta announced 2/26) A comprehensive System z offering for Enterprise BI Enterprise BI 67

68 Value proposition of zdw Qualities of Service Superior Quality Continuous Availability Security and Regulatory Compliance Scalability Backup and recovery Operational data and the ODS together means Reduced complexity Reduced cost Shared processes, tools, procedures Streamlined compliance and security Positioned for the future Web-based applications XML support Service Oriented Architecture (SOA) ziip and other specialty engines improve Total Cost of Ownership Better leverage System z skills and investment 68

69 Questions 69

70 Additional References My DB2 for z/os blog Articles in z/journal Magazine "Structure and Format Enhancements in DB2 9 for z/os" "Myth-Busting: DB2 Version 8 CPU Regression The Big Deal About Making Things Smaller: DB2 Compression ziiping Along Redbooks SG DB2 9 for z/os Technical Overview SG DB2 9 for z/os Performance Topics SG DB2 UDB for z/os Version 8: Everything You Ever Wanted to Know,... and More SG DB2 UDB for z/os Version 8 Performance Topics SG Best Practices for SAP BI using DB2 9 for zos SG Best Practices for SAP Business Information Warehouse on DB2 for zos V8 All available for download from

71 71

72 Session B10 Should Your Next Warehouse be on DB2 for z/os? Willie Favero Senior Certified Consulting IT Software Specialist Dynamic Warehousing on System z Swat Team IBM Silicon Valley Laboratory IBM Academic Initiative Ambassador for System z IBM Certified Database Administrator - DB2 Universal Database V8.1 for z/os IBM Certified Database Administrator DB2 9 for z/os IBM zchampion wfavero@attglobal.net 72