DB2 Temporal tables. Introduction. 19 April Rajesh Venkata Rama Mallina DB2 Z/OS DBA IBM

Size: px

Start display at page:

Download "DB2 Temporal tables. Introduction. 19 April Rajesh Venkata Rama Mallina DB2 Z/OS DBA IBM"

Clementine Jefferson
5 years ago
Views:

DB2 Temporal tables Rajesh Venkata Rama Mallina (vmallina@in.ibm.

In RDMS this is possible by way of temporal tables.

the data is valid for business). This paper caters to the explanation how all these can be implemented in DB2 for Z/OS and also learn about the features of temporal tables in DB2 10 and DB2 11.

1 DB2 Temporal tables Rajesh Venkata Rama Mallina DB2 Z/OS DBA IBM 19 April 2017 As part of data management scenarios, any update and deletion of data requires and saving old data called version data in tables. In RDMS this is possible by way of temporal tables. Temporal tables are the tables used to manage system time sensitive data (providing capability to query old snapshot of data for auditing) and business time sensitive data (providing period where the the data is valid for business). This paper caters to the explanation how all these can be implemented in DB2 for Z/OS and also learn about the features of temporal tables in DB2 10 and DB2 11. Introduction What is temporal tables? A temporal table is a table that records the period of time when a row is valid. Temporal Data Management in DB2 Provides automated feature that help you easily record and query past, current and future data conditions in tables. Types of temporal tables/temporal data management There are 3 ways of temporal data management System Period Management Application Period Management Bi-Temporal Management System Period Management: It provides automated feature that helps easily record and query current and past data conditions in our tables, which provides the capability to query past snapshot of data. Application Period Management: It provides feature to provide period where data is valid for business. Bi-Temporal Management: Copyright IBM Corporation 2017 Trademarks DB2 Temporal tables Page 1 of 13

2 developerworks ibm.com/developerworks/ It provides the combination of system period and application period data management. Note : Temporal tables were introduced in DB2 10 for z/os and enhanced in V11 and V12. System period temporal tables is for versioned data and system time sensitive data. Business period temporal tables for business time sensitive data. System period temporal tables 2.1 Implementation Implementation can be made in 3 steps 1. Create a system-period temporal table, CREATE TABLE statement with the following requirements a. Row begin timestamp column b. Row end timestamp column c. Transaction start Id timestamp column d. PERIOD SYSTEM_TIME/PERIOD SYSTEM TIME clause 2. Create a history table for system-period temporal table with same defination (DDL) excluding SYSTEM_TIME, row begin, row end, transaction start Id clauses. 3. ALTER TABLE ADD VERSIONING statement with the USE HISTORY TABLE clause to define system-period data versioning on the table. Example Step: 1 Create base table CREATE TABLE VXB5B1.TEMPOTEST ( ID INTEGER, C_DATA CHARACTER(12) FOR SBCS DATA, R_B_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN, R_E_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END, T_I_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS TRANSACTION START ID, PERIOD SYSTEM_TIME (R_B_TS,R_E_TS) ) IN DATABASE MALLIV; These three columns (row begin, row end and transaction start Id) should be NOT NULL columns and GENERATED ALWAYS columns. The system time consists of a pair of columns with system-maintained values that indicate the period of time when a row is valid. The begin column contains the timestamp value for when a row is created. The end column contains the timestamp value for when a row is updated or deleted. Transaction start Id specifies the timestamp value, whenever a row is inserted into the table or any column in the row is updated. Step: 2 Create history table DB2 Temporal tables Page 2 of 13

3 ibm.com/developerworks/ developerworks CREATE TABLE VXB5B1.TEMPOTEST_HIST ( ID INTEGER, C_DATA CHARACTER(12) FOR SBCS DATA, R_B_TS TIMESTAMP(12) NOT NULL, R_E_TS TIMESTAMP(12) NOT NULL, T_I_TS TIMESTAMP(12) NOT NULL ) IN DATABASE MALLIV; Step: 3 Add versioning use history table ALTER TABLE VXB5B1.TEMPOTEST ADD VERSIONING USE HISTORY TABLE VXB5B1.TEMPOTEST_HIS; Note: If history table is not created properly according to its norms set by DB2, then the history table cannot be successfully for the base table and the following error message is received. ALTER TABLE VXB5B1.TEMPOTEST ADD VERSIONING USE HISTORY TABLE VXB5B1.TEMPOTEST_HIST; DSNT408I SQLCODE = , ERROR: TABLE VXB5B1.TEMPOTEST_HIST WAS SPECIFIED AS A HISTORY TABLE, BUT THE TABLE DEFINITION IS NOT VALID FOR A HISTORY TABLE. REASON CODE = Data manipulation INSERT INSERT INTO VXB5B1.TEMPOTEST(ID,C_DATA) VALUES (1,'PHANI') When a row is inserted in system period temporal table, row begin timestamp & transaction ID timestamp coloumns will be updated with timestamp of inserted row. End row timestamp column will be updated with maximum timestamp value. Nothing will be updated in the history table. Column Base table History table ID 1 NA C_DATA PHANI NA R_B_TS NA R_E_TS NA T_I_TS NA UPDATE UPDATE VXB5B1.TEMPOTEST SET C_DATA='DEEPU' WHERE ID=1; When a row is updated in system period temporal table, row begin timestamp & transaction ID timestamp columns will be updated with timestamp of the updated row. End row timestamp column will be updated with maximum timestamp value. DB2 Temporal tables Page 3 of 13

4 developerworks ibm.com/developerworks/ Row before update will be copied into history table, and its row begin timestamp remains unchanged. End row timestamp column and transaction ID timestamp column will be updated with timestamp associated with update. Column Base table History table ID 1 1 C_DATA DEEPU PHANI R_B_TS R_E_TS T_I_TS DELETE DELETE FROM VXB5B1.TEMPOTEST WHERE ID = 1; When a row is deleted in system period temporal table, row moves into history table, and its row begin timestamp remains unchanged. End row timestamp column and transaction ID timestamp column will be updated with timestamp associated with delete. Column Base table History table ID NA 1 C_DATA NA DEEPU R_B_TS NA R_E_TS NA T_I_TS NA Note: DML activities can also be performed on history table. In order to maintain data consistency between various versions stored in the history table, no DML activities should be performed on history table. 2.3 Data retrieval During the execution of DELETE or UPDATE the versioned data (before image) will be moved to the history table. To retrieve the versioned data (history table data) along with existing data (base table data), SELECT query requires period specification (SYSTEM_TIME clause), which can be done in one of the following three ways as listed below:- DB2 Temporal tables Page 4 of 13

5 ibm.com/developerworks/ developerworks FOR SYSTEM_TIME AS OF... SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS FROM VXB5B1.TEMPOTEST FOR SYSTEM_TIME AS OF ' '; FOR SYSTEM_TIME FROM... TO... SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS FROM VXB5B1.TEMPOTEST FOR SYSTEM_TIME FROM ' ' TO ' ' FOR SYSTEM_TIME BETWEEN... AND... SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS FROM VXB5B1.TEMPOTEST FOR SYSTEM_TIME BETWEEN ' ' AND ' ' 3.4 Internal query transformation If the query contains SYSTEM_TIME clause as a predicate, the query internally converts into couple of SELECT queries (one on the base table and another one on the history table) joined with UNION ALL clause. DB2 generates implicit predicates for each query. Consider the below query with SYSTEM_TIME clause. SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS FROM VXB5B1.TEMPOTEST FOR SYSTEM_TIME FROM ' ' TO ' ' WHERE ID < 3 DB2 implicitly generated equivalent query as shown below. SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS FROM ( SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS FROM VXB5B1.TEMPOTEST WHERE R_B_TS < ' ' AND R_E_TS > ' ' UNION ALL SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS FROM VXB5B1.TEMPOTEST_HIS WHERE R_B_TS < ' ' AND R_E_TS > ' ' ) AS TEMPOTEST WHERE ID < 3 3. Application period temporal tables 3.1Implementation An application period temporal table has two user-maintained and NOT NULL Date or Timestamp without time zone columns denoting the period of time for which data is valid in the table. Both columns must be of the same datatype. Below DDL is an example to create an application period-temporal table. DB2 Temporal tables Page 5 of 13

6 developerworks ibm.com/developerworks/ CREATE TABLE VXB5B1.APPTEMPO (CUST_ID INTEGER,C_DATA CHARACTER(12) FOR SBCS DATA,BUS_ST_DT DATE NOT NULL,BUS_EN_DT DATE NOT NULL,PERIOD BUSINESS_TIME (BUS_ST_DT,BUS_EN_DT) ) IN DATABASE MALLIV 3.2 Data manipulation There is no implicitly updated columns for application period temporal tables. Columns have to be updated manually as per the business logic. INSERT INTO VX$B5B1.APPTEMPO(CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT) VALUES (1,'PHANI',' ',' '); PORTION OF BUSINESS_TIME clause With DB2 10 for z/os, to delete or update data in a portion of time on an application period temporal table, DELETE/UPDATE clauses need to be used with the FOR PORTION OF BUSINESS_TIME clause. Synatx and example is shown below. Syntax: DELETE FROM..FOR PORTION OF BUSINESS_TIME FROM value1 TO value2; UPDATE.. FOR PORTION OF BUSINESS_TIME FROM value1 TO value2; Example: 1 UPDATE VX$B5B1.APPTEMPO FOR PORTION OF BUSINESS_TIME FROM ' ' TO ' ' SET CUST_ID=9 WHERE CUST_ID< 4 Example: 2 DELETE FOR PORTION OF BUSINESS_TIME FROM ' ' TO ' ' WHERE CUST_ID< Data retrieval SELECT query requires period specification (BUSINESS_TIME clause) extract the data which is applicable for a specific period, which can be done in one of the 3 ways as below. FOR BUSINESS_TIME AS OF... SELECT CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT FOR BUSINESS_TIME AS OF ' ' ; FOR BUSINESS_TIME FROM... TO... DB2 Temporal tables Page 6 of 13

7 ibm.com/developerworks/ developerworks SELECT CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT FOR BUSINESS_TIME FROM ' ' TO ' ' ; FOR BUSINESS_TIME BETWEEN... AND... SELECT CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT FOR BUSINESS_TIME BETWEEN ' ' AND ' ' ; 3.4 Internal query transformation SELECT/INSERT/UPDATE statements may contain the BUSINESS_TIME clause. DB2 will implicitly convert the FOR clause to a WHERE clause predicate for such queries as shown in below examples. Example-1 SELECT CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT FOR BUSINESS_TIME AS OF ' ' WITH UR; DB2 implicity generated equivalent query as shown below. SELECT CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT WHERE BUS_ST_DT <= ' ' AND BUS_EN_DT > ' ' WITH UR; Example-2 DELETE FOR PORTION OF BUSINESS_TIME FROM ' ' TO ' ' WHERE CUST_ID< 4 DB2 implicitly generated equivalent query as shown below. DELETE FROM POLICY_ATT WHERE BUS_ST_DT < ' ' AND BUS_EN_DT > ' ' 3.5 Period overlapping BUSINESS_TIME WITHOUT OVERLAPS clause indicates that the values for the non-period columns and expressions of the index key for a row must be unique with respect to the time represented by the BUSINESS_TIME period for the row. DB2 enforces that multiple rows do not exist with the same key values for the columns or expressions of the index, with overlapping time periods. Below is the DDL for unique index creation with BUSINESS_TIME WITHOUT OVERLAPS clause. DB2 Temporal tables Page 7 of 13

8 developerworks ibm.com/developerworks/ CREATE UNIQUE INDEX VX$B5B1.APPTEMPOIX ON VX$B5B1.APPTEMPO (CUST_ID,BUSINESS_TIME WITHOUT OVERLAPS); Consider the below example, let's assume a row is present in the above table with CUST_ID=1. Now one another INSERT is trying to insert a new row with CUST_ID=1 with an overlapped time period between and as shown, which will eventually fail with duplicate key issue (SQLCODE=-803). CUST_ID C_DATA BUS_ST_DT BUS_EN_DT 1 RAVI INSERT INTO VX$B5B1.APPTEMPO (CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT) VALUES (1,'RAVI',' ',' '); DSNT408I SQLCODE = -803, ERROR: AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE APPT1LDS CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X' '. Note: If we try to add a BUSINESS _TIME WITHOUT OVERLAPS index on the table containing overlapped data for the application time period, then the CREATE INDEX will fail with SQLCODE = Advanced topics 4.1 Bi- Temporal tables BTTs manage both system time and business time, and combine all the capabilities of system-period and application-period temporal tables. This combination enables applications to manage the business validity of their data while DB2 keeps a full history of any updates and deletes. Every BTT is also an STT and an ATT. Example: CREATE TABLE VXB5B1.TEMPOTEST (ID INTEGER, C_DATA CHARACTER(12) FOR SBCS DATA, R_B_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN, R_E_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END, T_I_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS TRANSACTION START ID, PERIOD SYSTEM_TIME (R_B_TS,R_E_TS), BUS_ST_DT DATE NOT NULL, BUS_EN_DT DATE NOT NULL, PERIOD BUSINESS_TIME (BUS_ST_DT,BUS_EN_DT) )IN DATABASE MALLIV; 4.2 Temporal registers CURRENT TEMPORAL SYSTEM_TIME and CURRENT TEMPORAL BUSINESS_TIME are couple of special registers were newly introduced in DB2 11. The values in this registers can be set using SET statement as shown below. DB2 Temporal tables Page 8 of 13

ibm.com/developerworks/ developerworks SET CURRENT TEMPORAL SYSTEM_TIME = NULL SET CURRENT TEMPORAL BUSINESS_TIME =TIMESTAMP('2008

9 ibm.com/developerworks/ developerworks SET CURRENT TEMPORAL SYSTEM_TIME = NULL SET CURRENT TEMPORAL BUSINESS_TIME =TIMESTAMP(' ')+5 DAYS; SET CURRENT TEMPORAL BUSINESS_TIME ' '; SYSTIMESENSITIVE YES: If we specified this option during bind. when BINDing the application, DB2 will prepare two separate access path strategies for queries against system temporal tables. As shown in Figure, DB2 will then use the relevant access strategy depending on whether the CURRENT TEMPORAL SYSTEM_TIME register is set. This approach provides the best of both worlds, with applications able to access historical data when required but avoid unnecessary performance overheads when it is not. 4.3 Catalog information If base table has any history table then that table and its schema are recored in VERSIONING_SCHEMA and VERSIONING_TABLE columns of sysibm.systables catalog table, otherwise these columns are blank. SELECT CREATOR, NAME, TYPE, VERSIONING_SCHEMA,VERSIONING_TABLE FROM SYSIBM.SYSTABLES WHERE VERSIONING_TABLE <> ' ' WITH UR; CREATOR NAME TYPE VERSIONING_SCHEMA VERSIONING_TABLE VXB5B1 SPTT T VXB5B1 HIS_SPTT VXB5B1 HIS_SPTT H VXB5B1 SPTT VXB5B1 TEMPOTEST T VXB5B1 TEMPOTEST_HIS VXB5B1 TEMPOTEST_ H VXB5B1 TEMPOTEST If the entry is for history table, then VERSIONING_SCHEMA and VERSIONING_TABLE columns has corresponding base table schema and base table name. To avoid such records in result table, below query will help. DB2 Temporal tables Page 9 of 13

10 developerworks ibm.com/developerworks/ SELECT CREATOR, NAME, TYPE, VERSIONING_SCHEMA,VERSIONING_TABLE FROM SYSIBM.SYSTABLES WHERE VERSIONING_TABLE <> ' ' AND TYPE = 'T' WITH UR; CREATOR NAME TYPE VERSIONING_SCHEMA VERSIONING_TABLE VXB5B1 SPTT T VXB5B1 HIS_SPTT VXB5B1 TEMPOTEST T VXB5B1 TEMPOTEST_HIS PERIOD and DEFAULT columns of SYSCOLUMNS table will help to obtain SYSTEM_TIME and BUSINESS_TIME clause information. If DEFAULT column has 'X' means corresponding column is defined with the AS TRANSACTION START ID attribute. PERIOD column indicates whether the column is the start or the end of the period for a SYSTEM_TIME or BUSINESS_TIME period. S T B C blank Column is the start of period SYSTEM_TIME Column is the end of period SYSTEM_TIME Column is the start of period BUSINESS_TIME Column is the end of period BUSINESS_TIME Column is not used as either the start or the end of a period Below is the sample query to get above column information. SELECT TBCREATOR,TBNAME, NAME, PERIOD, DEFAULT FROM SYSIBM.SYSCOLUMNS WHERE PERIOD <> ' ' OR DEFAULT='X' WITH UR; TBCREATOR TBNAME NAME PERIOD DEFAULT VXB5B1 SPTT R_B_TS S Q VXB5B1 SPTT R_E_TS T R VXB5B1 SPTT T_I_TS X VXB5B1 APT_T BUS_ST_DT B N VXB5B1 APT_T BUS_EN_DT C N VXB5B1 TEMPOTEST R_B_TS S Q VXB5B1 TEMPOTEST R_E_TS T R VXB5B1 TEMPOTEST T_I_TS X VXB5B1 APPTEMPO BUS_ST_DT B N VXB5B1 APPTEMPO BUS_EN_DT C N 4.4 Differences System time Captures the time when changes happened to data inside DB2 Past to the present time DB2's physical view of time System validity (transaction time) Present data will be available in base table and past data will be available in history table. Business time Captures the time when changes happened to business data Past, present, and future time The application's logical view of time Business validity (valid time) Total data is present in base table only. There is no need of history table. DB2 Temporal tables Page 10 of 13

11 ibm.com/developerworks/ developerworks Supports queries such as what prices were stored in the database on June 30? Supports queries such as: What price policy was active on June 30? 4.4 System temporal recovery Both the base table and history table can be recovered to different points of recovery. But it is always recommended to recover both tables to a common point of recovery so as to maintain data consistency aross versions stored in both tables. For example RECOVER TABLESPACE (MALLIV.TEMPOTES DSNUM ALL) TABLESPACE (MALLIV.TEMP1KYW DSNUM ALL) TOLOGPOINT X'0D1BB9FC5CE6' VERIFYSET YES Note: VERIFYSET Specifies whether the RECOVER utility verifies that all related objects (can be History or LOB or XML tables) that are required for a point-in-time recovery are included in the RECOVER control statement. VERIFYSET is valid only for point-in-time recovery TORBA or TOLOGPOINT. 4.5 System Temporal Table restrictions 1. No utility operation can that deletes data from a system-period temporal table can be performed. These utilities include LOAD REPLACE, REORG DISCARD, and CHECK DATA DELETE YES. 2. Column name or table name of a system-period temporal table or a history table cannot be renamed. ALTER TABLE VXB5B1.TEMPOTEST RENAME COLUMN ID TO C_ID ; DSNT408I SQLCODE = -750, ERROR: THE SOURCE TABLE VXB5B1. TEMPOTEST CANNOT BE RENAMED OR ALTERED AS SPECIFIED 3. Schema (data type, check constraint, referential constraint, etc.) cannot be altered for a system-period temporal table or history table; however, a new column can be added to a system-period temporal table. 4. No clones can be defined on the system-period temporal table or the history table. 5. Both the system-period temporal table or history table have to be inside a single table tablespace. 6. No CHECK DATA utility with the options LOBERROR INVALIDATE, AUXERROR INVALIDATE, or XMLERROR INVALIDATE can be run on a system-period temporal table. The CHECK DATA utility will fail with return code 8 7. History table or its table space cannot be dropped explicitly. (If base table is dropped, then history table will implicitly get dropped) DROP TABLE VXB5B1.TEMPOTEST_HIS DSNT408I SQLCODE = -478, ERROR: DROP OR REVOKE ON OBJECT TYPE TABLE CANNOT BE PROCESSED BECAUSE OBJECT VXB5B1 TEMPOTEST OF TYPE TABLE IS DEPENDENT ON IT Note: None of the above restrictions are eliminated from DB2 10 to DB2 12 for z/os. 4.6 Temporal views Views can be created on temporal tables DB2 Temporal tables Page 11 of 13

12 developerworks ibm.com/developerworks/ EX: CREATE VIEW TEMO_VIEW AS (SELECT ID,C_DATA,R_B_TS,R_E_TS FROM VX$B5B1.BITEMPOTEST) With V11 can use BUSINESS_TIME clause in DELETE and UPDATE statements on temporal table based views EX: UPDATE TEMO_VIEW FOR PORTION OF BUSINESS_TIME FROM ' ' TO ' ' SET ID = ID + 1; 4.7 Inclusive/exclusive period With DB2 V12, EXCLUSIVE/INCLUSIVE keywords are implicit check constraints which ensure the relationship of the value of end-column-name to the value of begin-column-name as follows EXCLUSIVE: Specifies that the value of the end column for the BUSINESS_TIME is not included for the row inserted (or) modified. The BUSINESS_TIME period is defined as begin-column-name (inclusive) and end-column-name (exclusive). INCLUSIVE: Specifies that the value of the end column is included in the period. The BUSINESS_TIME period is defined as inclusive-inclusive. 5. Bibliography DB2 Temporal tables Page 12 of 13

ibm.com/developerworks/ developerworks About the author Rajesh Venkata Rama Mallina Rajash has a bachelor's in electronics and communication engineering and then working in in IBM as DB2 DBA.

13 ibm.com/developerworks/ developerworks About the author Rajesh Venkata Rama Mallina Rajash has a bachelor's in electronics and communication engineering and then working in in IBM as DB2 DBA. He is a certified DB2 Database Associate. Copyright IBM Corporation 2017 ( Trademarks ( DB2 Temporal tables Page 13 of 13

DB2 Archive tables. Introduction. DDL Operations. 18 April Rajesh Venkata Rama Mallina DB2 Z/OS DBA IBM

DB2 Archive tables. Introduction. DDL Operations. 18 April Rajesh Venkata Rama Mallina DB2 Z/OS DBA IBM DB2 Archive tables Rajesh Venkata Rama Mallina (vmallina@in.ibm.com) DB2 Z/OS DBA IBM 18 April 2017 This paper will help in understanding the concepts of archive tables which includes its creation, maintenance