Temporal DB Features in Oracle

Transcription

1 Temporal DB Features in Oracle

2 Introduction Consider the following: a table Orders with the definition Create table Orders( order_number int not null primary key, product_id int not null, qty int not null, order_date date ); We would like to know what were the periods between orders related to each product How would you write such a query?

3 Introduction SELECT product_id, order_date, LAG (order_date,1) OVER (PARTITION BY product_id ORDER BY order_date) AS prev_order_date FROM orders order by 1, 2;

4 Introduction VT temporal support in Oracle enables you to associate a valid time dimension with a table and to have data be visible depending on its time-based validity, as determined by the start and end dates or timestamps of the period for which a given record is considered valid. VT is typically used with Oracle Flashback technology, to perform AS OF and VERSIONS BETWEEN queries that specify the valid time period. You can also use the DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time procedure to specify a option for the visibility of table data.

5 Introduction Some scenarios where valid time temporal support can be useful include: Information Lifecycle Management (ILM) and any other application where it is important to know when certain data became valid (from the application's perspective) and when it became invalid (if it ever did) Data correction where incorrect data needs to be retained and marked with the period when it was considered valid, and where the correct data needs to be visible as currently valid

6 Oracle Temporal Validity Support typically used with Oracle Flashback Technology for queries that specify the valid time period in AS OF and VERSIONS BETWEEN clauses. You can also use the DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME procedure (described in Oracle Database PL/SQL Packages and Types Reference) to specify an option for the visibility of table data: all table data (the default), data valid at a specified time, or currently valid data within the valid time period at the session level

7 Flashback Technology Flashback technology must be used when a logical corruption occurs in the Oracle database, and you need to recover data quickly and easily. As with human errors, it is difficult to identify the objects and rows that are affected by an erroneous transaction. With flashback technology, you can diagnose how errors are introduced into the database, and then you can repair the damage. You can view the transactions that have contributed to specific row modifications, view the entire set of versions of a given row during some time period, or just view data as it appeared at a specific time in the past.

8 Flashback - DB Admin Features Flashback Database: restore the entire database to a specific point-in-time, using Oracle-optimized flashback logs, rather than via backups and forward recovery. Flashback Table: easily recover tables to a specific point-in-time, useful when a logical corruption is limited to one or a set of tables instead of the entire database. Flashback Drop: recover an accidentally dropped table. It restores the dropped table, and all of its indexes, constraints, and triggers, from the Recycle Bin (a logical container of all dropped objects). Flashback Transaction: undo the effects of a single transaction, and optionally, all of its dependent transactions. via a single PL/SQL operation or by using an Enterprise Manager wizard.

9 Flashback - App Dev Features Flashback Transaction Query: see all the changes made by a specific transaction, useful when an erroneous transaction changed data in multiple rows or tables. Flashback Query: query any data at some point-in-time in the past. This powerful feature can be used to view and logically reconstruct corrupted data that may have been deleted or changed inadvertently. Flashback Versions Query: retrieve different versions of a row across a specified time interval instead of a single point-in-time.

10 Flashback Technology Flashback Database uses the flashback logs to perform flashback. Flashback Drop uses the recycle bin. All other techniques use undo data. Not all flashback features modify the database. Some are simply methods to query other versions of data. Those are tools for you to use to investigate a problem and aid in recovery. The results of those flashback queries can help you do one of these two things: Determine which type of database-modifying flashback operation to perform to fix the problem. Feed the result set of these queries into an INSERT, UPDATE, or DELETE statement that enables you to easily repair the erroneous data.

11 Flashback Benefits Flashback technology is a revolutionary advance in recovery Traditional recovery techniques are slow Entire database or file has to be restored, not just the incorrect data Every change in the database log must be examined Flashback is fast Changes are indexed by row and by transaction Only the changed data is restored Flashback commands are easy No complex multi-step procedures

12 Flashback Technology Object Level Scenario Examples Flashback Technology Uses Affects Data Database Truncate table; Undesired multitable changes made Database Flashback logs TRUE Table Drop table Drop Recycle bin TRUE Update with the wrong WHERE clause Compare current data with data from the past Table Undo data TRUE Query Undo data FALSE Compare versions of a row Version Undo data FALSE Tx Investigate several historical states of data Transaction Undo data FALSE

13 Flashback Database you can quickly bring your database to an earlier point in time by undoing all the changes that have taken place since that time. This operation is fast because you do not need to restore backups. You can use this feature to undo changes that have resulted in logical data corruptions. If you have experienced a loss of media or physical corruption in your database, then you must use traditional recovery methods

14 Flashback Database The Flashback Database operation: Works like a rewind button for the database Can be used in cases of logical data corruptions made by users Users generate errors. The database is corrupted. You press the rewind button. The database is rewound.

15 Flashback Database Reduces Restore Time Incomplete Recovery Generate logs Restore files Repaired database Backup User error Apply logs forward Flashback Database Flashback logs User error Repaired database Backup Apply Flashback logs backward

16 Configuring Flashback Database 1. Configure the flash recovery area 2. Set the retention target 3. Enable Flashback Database SQL> ALTER SYSTEM SET DB_FLASHBACK_RETENTION_TARGET=2880 SCOPE=BOTH; SQL> ALTER DATABASE FLASHBACK ON;

17 Monitoring Flashback Database Adjust the flash recovery area disk quota: SQL> SELECT estimated_flashback_size, flashback_size FROM V$FLASHBACK_DATABASE_LOG; Determine the current flashback window: SQL> SELECT oldest_flashback_scn, oldest_flashback_time FROM V$FLASHBACK_DATABASE_LOG; Monitor logging in the Flashback Database logs: SQL> SELECT * FROM V$FLASHBACK_DATABASE_STAT;

18 Flashback Database Examples RMAN> FLASHBACK DATABASE TO TIME = > TO_DATE(' :00:00', > 'YYYY-MM-DD HH24:MI:SS'); RMAN> FLASHBACK DATABASE TO SCN=23565; RMAN> FLASHBACK DATABASE > TO SEQUENCE=223 THREAD=1; SQL> FLASHBACK DATABASE TO TIMESTAMP(SYSDATE-1/24); SQL> FLASHBACK DATABASE TO SCN 53943;

19 Flashback Drop and Rec. Bin RECYCLEBIN=ON DROP TABLE employees; Mistake was made. FLASHBACK TABLE employees TO BEFORE DROP;

20 Flashback Drop In previous releases of the Oracle database, if you dropped a table by mistake, you had to recover the database to a prior time to recover the dropped table. This procedure was often time consuming and resulted in loss of work of other transactions. Oracle Database 10g introduced the Flashback Drop feature, which you can use to undo the effects of a DROP TABLE statement without having to use point-in-time recovery. The RECYCLEBIN initialization parameter is used to control whether the Flashback Drop capability is turned ON or OFF. If the parameter is set to OFF, then dropped tables do not go into the recycle bin. If this parameter is set to ON, the dropped tables go into the recycle bin and can be recovered. By default, RECYCLEBIN is set to ON.

21 Flashback Drop Use the FLASHBACK TABLE... TO BEFORE DROP command to recover a table and all of its possible dependent objects from the recycle bin. You can specify either the original name of the table or the system-generated name assigned to the object when it was dropped. If you specify the original name, and if the recycle bin contains more than one object of that name, then the object that was moved to the recycle bin most recently is recovered first (LIFO: last in, first out). If you want to retrieve an older version of the table, you can specify the system-generated name of the table that you want to retrieve, or issue additional FLASHBACK TABLE... TO BEFORE DROP statements until you retrieve the table you want.

22 Flashback Drop If a new table of the same name has been created in the same schema since the original table was dropped, then an error is returned unless you also specify the RENAME TO clause. When you flash back a dropped table, the recovered indexes, triggers, and constraints keep their recycle bin names. Therefore, it is advisable to query the recycle bin and DBA_CONSTRAINTS before flashing back a dropped table. In this way, you can rename the recovered indexes, triggers, and constraints to more usable names.

23 Querying the Recycle Bin SELECT owner, original_name, object_name, type, ts_name, droptime, related, space FROM dba_recyclebin WHERE can_undrop = 'YES'; SELECT original_name, object_name, type, ts_name, droptime, related, space FROM user_recyclebin WHERE can_undrop = 'YES'; SQL> SHOW RECYCLEBIN

24 Flashback Query To use Oracle Flashback Query, use a SELECT statement with an AS OF clause. Oracle Flashback Query retrieves data as it existed at some time in the past. The query explicitly references a past time through a timestamp or System Change Number (SCN). It returns committed data that was current at that point in time. Uses of Oracle Flashback Query include: Recovering lost data or undoing incorrect, committed changes. For example, if you mistakenly delete or update rows, and then commit them, you can immediately undo the mistake. Comparing current data with the corresponding data at some time in the past.

25 Flashback Query For example, you can run a daily report that shows the change in data from yesterday. You can compare individual rows of table data or find intersections or unions of sets of rows. Simplifying application design by removing the need to store some kinds of temporal data. Oracle Flashback Query lets you retrieve past data directly from the database. Providing self-service error correction for an application, thereby enabling users to undo and correct their errors.

26 Flashback Query CREATE TABLE my_emp( empno NUMBER, last_name VARCHAR2(30), start_time TIMESTAMP, end_time TIMESTAMP, PERIOD FOR user_valid_time (start_time, end_time)); INSERT INTO my_emp VALUES (100, 'Ames', '01-Jan-10', '30-Jun-11'); INSERT INTO my_emp VALUES (101, 'Burton', '01-Jan-11', '30-Jun-11'); INSERT INTO my_emp VALUES (102, 'Chen', '01-Jan-12', null);

27 Table with Valid Time Support -- Valid Time Queries AS OF PERIOD FOR queries: -- Returns only Ames. SELECT * from my_emp AS OF PERIOD FOR user_valid_time TO_TIMESTAMP('01-Jun-10'); -- Returns Ames and Burton, but not Chen. SELECT * from my_emp AS OF PERIOD FOR user_valid_time TO_TIMESTAMP('01-Jun-11'); -- Returns no one. SELECT * from my_emp AS OF PERIOD FOR user_valid_time TO_TIMESTAMP( '01-Jul-11'); -- Returns only Chen. SELECT * from my_emp AS OF PERIOD FOR user_valid_time TO_TIMESTAMP('01-Feb-12');

28 Table with Valid Time Support -- VERSIONS PERIOD FOR... BETWEEN queries: -- Returns only Ames. SELECT * from my_emp VERSIONS PERIOD FOR user_valid_time BETWEEN TO_TIMESTAMP('01-Jun-10') AND TO_TIMESTAMP('02-Jun-10'); -- Returns Ames and Burton. SELECT * from my_emp VERSIONS PERIOD FOR user_valid_time BETWEEN TO_TIMESTAMP('01-Jun-10') AND TO_TIMESTAMP('01-Mar-11'); -- Returns only Chen. SELECT * from my_emp VERSIONS PERIOD FOR user_valid_time BETWEEN TO_TIMESTAMP('01-Nov-11') AND TO_TIMESTAMP('01-Mar-12'); -- Returns no one. SELECT * from my_emp VERSIONS PERIOD FOR user_valid_time BETWEEN TO_TIMESTAMP('01-Jul-11') AND TO_TIMESTAMP('01-Sep-11');

29 Other Usage Examples You can create a view that refers to past data by using the AS OF clause in the SELECT statement that defines the view. If you specify a relative time by subtracting from the current time on the database host, the past time is recalculated for each query. For example: CREATE VIEW hour_ago AS SELECT * FROM employees AS OF TIMESTAMP (SYSTIMESTAMP - INTERVAL '60' MINUTE);

30 Other Usage Examples You can use the AS OF clause in self-joins, or in set operations such as INTERSECT and MINUS, to extract or compare data from two different times. You can store the results by preceding Oracle Flashback Query with a CREATE TABLE AS SELECT or INSERT INTO TABLE SELECT statement. For example, the following query reinserts into table employees the rows that existed an hour ago: INSERT INTO employees (SELECT * FROM employees AS OF TIMESTAMP (SYSTIMESTAMP - INTERVAL '60' MINUTE) ) MINUS SELECT * FROM employees);

31 Add Valid Time Support To add Temporal Validity Support to an existing table without explicitly adding columns, use the ALTER TABLE statement with the ADD PERIOD FOR clause. For example, if the CREATE TABLE statement did not create the START_TIME and END_TIME columns, you could use this statement: ALTER TABLE my_emp ADD PERIOD FOR user_valid_time; The preceding statement adds two hidden columns to the table MY_EMP: USER_VALID_TIME_START and USER_VALID_TIME_END. You can insert rows that specify values for these columns, but the columns do not appear in the output of the SQL*Plus DESCRIBE statement, and SELECT statements show the data in those columns only if the SELECT list explicitly includes the column names.

32 Add Valid Time Support This uses Temporal Validity Support for data change in the table created previously. The initial record for employee 103 has the last name Davis. Later, the employee changes his or her last name to Smith. The END_TIME value in the original row changes from NULL to the day before the change is to become valid. A new row is inserted with the new last name, the appropriate START_TIME value, and END_TIME set to NULL to indicate that it is valid until set otherwise.

33 Add Valid Time Support -- Add a record for Davis. INSERT INTO my_emp VALUES (103, 'Davis', '01-Jan-12', null); -- To change employee 103's last name to Smith, first set an end date for the record with the old name. UPDATE my_emp SET end_time = '01-Feb-12' WHERE empno = 103; -- Insert a record for employee 103, specifying the new last name, the valid start date, and null for the valid end date. -- After the INSERT statement, there are two records for #103 (Davis and Smith). INSERT INTO my_emp VALUES (103, 'Smith', '02-Feb-12', null);

34 Add Valid Time Support -- What's the valid information for employee 103 as of today? SELECT * from my_emp AS OF PERIOD FOR user_valid_time SYSDATE WHERE empno = 103; -- What was the valid information for employee 103 as of a specified date? -- First, as of a date after the change to Smith. SELECT * from my_emp AS OF PERIOD FOR user_valid_time TO_TIMESTAMP('01-Jul-12') WHERE empno = 103; -- Next, as of a date before the change to Smith. SELECT * from my_emp AS OF PERIOD FOR user_valid_time TO_TIMESTAMP('20-Jan-12') WHERE empno = 103;

35 Flashback Version Query Oracle Flashback Version Query is used to retrieve the different versions of specific rows that existed during a given time interval. A new row version is created whenever a COMMIT statement is executed. Specify Oracle Flashback Version Query using the VERSIONS BETWEEN clause of the SELECT statement. The syntax is: VERSIONS {BETWEEN {SCN TIMESTAMP} start AND end} where start and end are expressions representing the start and end, respectively, of the time interval to be queried. The time interval includes (start and end). Oracle Flashback Version Query returns a table with a row for each version of the row that existed at any time during the specified time interval. Each row in the table includes pseudocolumns of metadata about the row version. This information can reveal when and how a particular change (perhaps erroneous) occurred to the database.

36 Flashback Version Query VERSIONS_STARTSCN VERSIONS_STARTTIME Starting System Change Number (SCN) or TIMESTAMP when the row version was created. This pseudocolumn identifies the time when the data first had the values reflected in the row version. Use this pseudocolumn to identify the past target time for Oracle Flashback Table or Oracle Flashback Query. If this pseudocolumn is NULL, then the row version was created before start. VERSIONS_ENDSCN VERSIONS_ENDTIME SCN or TIMESTAMP when the row version expired. If this pseudocolumn is NULL, then either the row version was current at the time of the query or the row corresponds to a DELETE operation.

37 Flashback Version Query VERSIONS_XID Identifier of the transaction that created the row version. VERSIONS_OPERATION Operation performed by the transaction: I for insertion, D for deletion, or U for update. The version is that of the row that was inserted, deleted, or updated; that is, the row after an INSERT operation, the row before a DELETE operation, or the row affected by an UPDATE operation. For user updates of an index key, Oracle Flashback Version Query might treat an UPDATE operation as two operations, DELETE plus INSERT, represented as two version rows with a D followed by an I VERSIONS_OPERATION.

38 Flashback Version Query SELECT versions_startscn, versions_starttime, versions_endscn, versions_endtime, versions_xid, versions_operation, name, salary FROM employees VERSIONS BETWEEN TIMESTAMP TO_TIMESTAMP(' :00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_TIMESTAMP(' :00:00', 'YYYY-MM-DD HH24:MI:SS') WHERE name = 'JOE';

39 Flashback Transaction Query Oracle Flashback Transaction Query is used to retrieve metadata and historical data for a given transaction or for all transactions in a given time interval. Oracle Flashback Transaction Query queries the static data dictionary view FLASHBACK_TRANSACTION_QUERY. The column UNDO_SQL shows the SQL code that is the logical opposite of the DML operation performed by the transaction. You can usually use this code to reverse the logical steps taken during the transaction. However, there are cases where the SQL_UNDO code is not the exact opposite of the original transaction. For example, a SQL_UNDO INSERT operation might not insert a row back in a table at the same ROWID from which it was deleted.

40 Flashback Transaction Query The following statement queries the FLASHBACK_TRANSACTION_QUERY view for transaction information, including the transaction ID, the operation, the operation start and end SCNs, the user responsible for the operation, and the SQL code that shows the logical opposite of the operation: SELECT xid, start_scn, commit_scn, operation, logon_user, undo_sql FROM flashback_transaction_query WHERE xid = HEXTORAW(' D'); D DELETE HR insert into "HR"."EMP" ("EMPNO","EMPNAME","SALARY") values ('111','Mike','655'); D INSERT HR delete from "HR"."DEPT" where ROWID = 'AAAKD4AABAAAJ3BAAB'; D UPDATE HR update "HR"."EMP" set "SALARY" = '555' where ROWID = 'AAAKD2AABAAAJ29AAA'; D BEGIN HR 4 rows selected

41 Oracle References rials/obe/db/12c/r1/ilm/temporal/temporal.html 1/b28424/adfns_flashback.htm

42 Temporal Tables in MS Sql Server

43 Introduction Temporal tables are a new feature of SQL Server Temporal tables, also named system-versioned tables, allow SQL Server to automatically keep history of the data in the table. A system-versioned table allows you to query updated and deleted data, while a normal table can only return the current data. For example, if you update a column value from 5 to 10, you can only retrieve the value 10 in a normal table. A temporal table also allows you to retrieve the old value 5. This is accomplished by keeping a history table. This history table stores the old data together with a start and end data to indicate when the record was active.

44 Introduction

45 Introduction Temporal tables or system-versioned tables is an assertion table, meaning that it captures the lifetime of a record based on the physical dates the record was removed or updated. Temporal tables currently do not support versioning, meaning the versioning of records based on logical dates. For example, suppose you have a table keeping product prices. If you update the price at 12PM using an UPDATE statement, the temporal table will keep the history of the old price until 12PM of that day. Starting from 12PM, the new price is valid. However, what if the price change was actually meant to start from 1PM (a logical change)? This means you have to time your update statement perfectly in order to make it work and you should have executed the UPDATE statement at 1PM instead of 12PM.

46 System-versioned Table Creation For a new created temporal table, the following apply: A primary key must be defined Two columns must be defined to record the start and end date with a data type of datetime2. If needed, these columns can be hidden using the HIDDEN flag. These columns are called the SYSTEM_TIME period columns. INSTEAD OF triggers are not allowed. AFTER triggers are only allowed on the current table. In-memory OLTP cannot be used There are also some limitations: Temporal and history table cannot be FILETABLE The history table cannot have any constraints INSERT and UPDATE statements cannot reference the SYSTEM_TIME period columns Data in the history table cannot be modified

47 System-versioned Table Creation CREATE TABLE dbo.testtemporal (ID int primary key,a int,b int,c AS A * B,SysStartTime datetime2 GENERATED ALWAYS AS ROW START NOT NULL,SysEndTime datetime2 GENERATED ALWAYS AS ROW END NOT NULL,PERIOD FOR SYSTEM_TIME (SysStartTime,SysEndTime)) WITH(SYSTEM_VERSIONING = ON (HISTORY_TABLE=dbo.TestTemporal_History)); If you don't specify a name for the history table, SQL Server will automatically generate one of the following structure: dbo.mssql_temporalhistoryfor_xxx, where xxx is the object id of the main table.

48 System-versioned Table Creation The history table has an identical set of columns, but with all constraints removed. It also has its own set of indexes and statistics. Creating your own indexes such as a clustered columnstore index on the history table can greatly improve performance.

49 System-versioned Table Operations -- Initial Load INSERT INTO dbo.testtemporal(id, A, B) VALUES (1,2,3),(2,4,5),(3,0,1); SELECT * FROM dbo.testtemporal;

50 System-versioned Table Operations -- Modify Data DELETE FROM dbo.testtemporal WHERE ID = 2; UPDATE dbo.testtemporal SET A = 5 WHERE ID = 3;

51 Alter Table to System-versioned SQL Server allows you to convert an existing table into a temporal table. In order to perform this task, you need to make sure that a primary key exists on the table, or to create one. Then the table has to be altered with two datetime2 data type columns and also the option GENERATED ALWAYS AS ROW START/END has to be applied with PERIOD FOR SYSTEM_TIME (namefrom, nameto). Both options have to be completed with ALTER command. The second ALTER command enables the SYSTEM_VERSIONING property, and optionally provide a name for the HISTORY_TABLE property

52 Alter Table to System-versioned CREATE TABLE Department_Exist ( DeptID int NOT NULL PRIMARY KEY CLUSTERED, DeptName varchar(50) NOT NULL, ManagerID INT NULL, ParentDeptID int NULL ) GO ALTER TABLE dbo.department_exist ADD SysStartTime datetime2 GENERATED ALWAYS AS ROW START CONSTRAINT DF_Department_Exist_SysStartTime DEFAULT SYSUTCDATETIME() NOT NULL, SysEndTime datetime2 GENERATED ALWAYS AS ROW END CONSTRAINT DF_Department_Exist_SysEndTime DEFAULT CONVERT (DATETIME2, ' :59:59') NOT NULL, PERIOD FOR SYSTEM_TIME (SysStartTime,SysEndTime) GO ALTER TABLE dbo.department_exist SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.department_existhistory)) GO