Db2 Analytics Accelerator V5.1 What s new in PTF 5

Transcription

1 Ute Baumbach, Christopher Watson IBM Boeblingen Laboratory Db2 Analytics Accelerator V5.1 What s new in PTF 5

2 Legal Disclaimer IBM Corporation All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. 2

3 Db2 Analytics Accelerator V5.1 improvements in PTF 5 PTF 5 improvements (GA mid July 2017) Multiple-Row Insert into AOTs Populate AOTs faster from external tools/sources Add Column Support (Without Incremental Update) Eliminate need to reload tables after adding a column Federation Enable read access to all data in one Accelerator through a single Db2 subsystem 3

4 Multiple-row insert for Accelerator-only tables (AOTs) Support of multiple-row INSERTs for accelerator-only tables Allows faster inserts into accelerator-only table through JDBC, ODBC or local SQL applications Now GA state Db2 11 with Db2 APAR/PTF PI66573/ UI45707 Db2 12 support planned for later this year From Db2 INSERT statement: Performance numbers from test in Lab: Multiple-row insert is ~66 times faster than single row insert JDBC program with batch size of rows for multiple-row insert Use cases: Populating AOTs using INSERT statements from external sources ETL processing outside Db2, then insert result into AOT Populating AOTs from analytics tools such as IBM Campaign or Data Stage 4

5 Example for Multiple-row insert using JDBC 5

6 Add Column support After a column has been added to a Db2 table the column can be added to the associated table on the Accelerator by executing a synchronization operation No need any more to remove the table from the Accelerator, re-add it and re-load it again Supported table types: Accelerator-shadow tables PTF 5: Without Incremental Upate enabled only PTF 6: With or without Incremental Update enabled Accelerator-archive tables 6 Supported Schema Changes ALTER TABLE ADD COLUMN (adding a nullable column) ALTER TABLE ADD COLUMN... NOT NULL WITH DEFAULT (adding a non-nullable column with a default value) Default values are supported as long as they are explicit. Default values like e.g. TIMESTAMP WITH DEFAULT; are not supported, as there is no way to determine the Db2 side value at the time of the ALTER statement in this case Hexadecimal character strings are not supported as default value ALTER TABLE ALTER COLUMN <name> SET DATA TYPE (limited to increase length of VARCHAR)

7 Interfaces for the Add Column support New stored procedure ACCEL_SYNCHRONIZE_SCHEMA to synchronize the schema and alter the specified table(s) on the Accelerator New Accelerator Studio controls: 7

8 Federation support Enable read access to all data in one accelerator through a single Db2 system Prerequisite: Data from different Db2 systems is loaded into the same accelerator V5.1 PTF 5 and Db2 11 with APARs/PTFs PI77826/UI48595 and PI76645/ UI48595 installed Db2 12 support planned later this year SELECT DB2R. DB2O1 DB2O2. Accelerator. 8

9 Federation use case examples Data is stored in different Db2 subsystems and might be separated by LoB or by Region. Consolidate this data into a single DWH on z Systems by creating a federated view Data for multiple tenants are stored in multiple Db2 subsystems (each has the same logical data model). Consolidate this data into a single federated system and build union all views of all tenants A Db2 subsystem represents a data lake that is maintained by Db2 Analytics Accelerator Loader from sources external to Db2 for z/os (Master Db2 subsystem). Then one or more referencing Db2 subsystems can access the data. 9

10 Federation scenario Terminology: Owning Db2 (DB2O) Db2 that has read/write access to a table on the Accelerator. Table was added by DB2O to the Accelerator Referencing Db2 (DB2R) Db2 that references, in RO mode, a (remote) table on the Accelerator that was added by DB2O to the Accelerator DBA of DB2O grants privileges to select from tables to DB2R 10 New Accelerator stored procedure DBA of DB2R creates references to tables for that privileges were granted from DB2O Another new Accelerator stored procedure References created as AOTs to get a proxy within Db2 catalog tables of DB2R User of DB2R executes a query on the Accelerator that joins tables from both DB2O and DB2R (T1 and T2_O) Uses referencing AOTs in DB2R for DB2O tables in query SELECT on T1 and T2_O Accelerator Table T1 DB2R AOT T2_O DB2O Table T2

11 Federation related new stored procedures Grants another Db2 for z/os subsystem (DB2R) the privilege to select from acceleratorshadow or accelerator-only tables that this subsystem (DB2O) has added to the specified accelerator (not a Db2 GRANT) ACCEL_GRANT_TABLES_REFERENCE (ACCEL1, DB2R, Set<Accelerator-table>) Creates a referencing accelerator-only table (AOT) for each table listed in the specified table set that allows this Db2 for z/os subsystem (DB2R) to reference an accelerator-shadow or accelerator-only table created in another subsystem (DB2O) from a query running in this subsystem. ACCEL_CREATE_REFERENCE_TABLES (ACCEL1, DB2O, Set<Accelerator-table-of-db2o>) No data duplication on Accelerator, just references (referencing AOT in DB2R and synonym on the Accelerator to point to the original DB2O table on Accelerator) Removes a referencing AOT on DB2R for each table that is referenced from DB2O and listed in the specified table set. Do not use SQL DROP for referencing AOTs. ACCEL_REMOVE_REFERENCE_TABLES (ACCEL1, DB2O, Set<Accelerator-table-of-db2o>) Revokes the privilege to select from accelerator-shadow or accelerator-only tables that this subsystem (DB2O) has added to the specified accelerator from another Db2 for z/os subsystem (DB2R) 11 ACCEL_REVOKE_TABLES_REFERENCE (ACCEL1, DB2R, Set<Accelerator-table>)

12 Federation: More technical details The SELECT privilege for the referencing AOTs is managed/checked in the referencing Db2 subsystem (DB2R) Table metadata for referencing AOTs is required in DB2R for query rewrite and eligibility analysis Creation of referencing AOTs inserts entries in SYSIBM.SYSTABLES and SYSACCEL.SYSACCELERATEDTABLES on DB2R Db2 catalog changes New pseudo-catalog table SYSACCEL.SYSACCELERATEDTABLESAUTH Contains one row for each accelerator-shadow or accelerator-only table on the owning Db2 (DB2O) for which a grant exists that allows another subsystem sharing the same accelerator to access the table New column REMOTELOCATION in SYSACCEL.SYSACCELERATEDTABLES Contains the Db2 location of the owner table being referenced by a referencing AOT on DB2R. If the table schema has been changed (ADD COLUMN) on DB2O and the change has been synchronized to the Accelerator a reference to that table must be recreated on DB2R. 12

13 Federation: More technical details (contd.) If the referencing AOT on DB2R references a loaded shadow-table on DB2O and this table is enabled for replication later, queries using the referencing AOT on DB2R get a negative SQLcode while the table is in Initial-Load-Pending or Initial-Load in progress state on DB2O Limitation: If the referencing AOT on DB2R references an archived table (HPSS) on DB2O, then only the active partitions can be queried. The archived partitions cannot be queried CURRENT GET_ACCEL_ARCHIVE=YES on DB2R is ignored for this table The referencing side DB2R can not write to a referencing AOT Referencing AOT on DB2R cannot be loaded by Db2 Analytics Accelerator Loader. The referenced accelerator-shadow or accelerator-only table on DB2O has to be loaded. Graphical dialogs in Accelerator Studio to grant and revoke table references, and to create and remove reference tables New icon for federated table Referencing AOT = Federated table 13

14 Federation supports Workload Balancing and High-Availability In a high-availability setup, both Db2 subsystems are connected to at least two shared Accelerators. DB2O loads the tables into both Accelerators (ACCEL1 and ACCEL2). In DB2R a single referencing AOT points to both Accelerators (ACCEL1 and ACCEL2): DB2O: ACCEL_GRANT_TABLES_REFERENCE (ACCEL1, DB2R, Set<Accelerator-table>) ACCEL_GRANT_TABLES_REFERENCE (ACCEL2, DB2R, Set<Accelerator-table>) DB2R: ACCEL_CREATE_REFERENCE_TABLES (ACCEL1, DB2O, Set<Accelerator-table-of-db2o>) ACCEL_CREATE_REFERENCE_TABLES (ACCEL2, DB2O, Set<Accelerator-table-of-db2o>) 14

15