SQL Server: Temporal Database Support

Transcription

1 Pasquale SQL Server: Temporal Database Support

2 Sponsor

3 Pasquale Database modelling Troubleshooting Performance tuning Database corruption Data Analysis

4 Agenda

5 - a database with built-in support for handling and archiving old versions of modified data over the time. - involves 2 tables: I. the main Temporal Table II. a History Table

6 A temporal table is a table: - With a PERIOD definition - Contains system columns with a datatype of DATETIME2 - Has an associated history table into which the system records all prior versions of each record with their period of validity. The value of each record at any point in time can be determined

7 Historical data may be critical to business success Traditional databases fail to provide required insights Time Travel Data Audit Complex, expensive, limited, inflexible, inefficient No change in programming model Provides new insights Slowly Changing Dimensions Repair record-level corruptions Reproducing financial reports, invoices and statements (with the correct data for the date of issue of the document) Analyzing trends

8 No change in programming model New insights CREATE temporal TABLE PERIOD FOR SYSTEM_TIME ALTER regular_table TABLE ADD PERIOD DDL INSERT / BULK INSERT UPDATE DELETE MERGE DML SELECT * FROM temporal Querying Temporal Querying FOR SYSTEM_TIME AS OF FROM..TO BETWEEN..AND CONTAINED IN

9 Temporal table (actual data) History Table * Old versions Update */ Delete * Insert / Bulk Insert

10 Temporal table (actual data) History Table * Include Historical Version Regular queries (current data) Temporal Queries * (Time travel,etc.)

11 Department (current) DepNum DepName MngrID From To A001 Marketing A002 Sales Department (history) DepNum DepName MngrID From To A001 Marketing A002 Sales A003 Consulting A003 Consulting

12 Department (current + history) DepNum DepName MngrID A001 Marketing 5 A001 Marketing 6 A002 Sales 2 A002 Sales 5 A003 Consulting 6 A003 Consulting 10 A001 A002 A003 A001 A002 A period of validity current time

13 Department (current + history) Get actual row versions DepNum DepName MngrID A001 Marketing 5 A001 Marketing 6 A002 Sales 2 A002 Sales 5 A003 Consulting 6 A003 Consulting 10 A001 A002 A003 A001 A002 A period of validity current time SELECT * FROM Department

14 Department (current + history) AS OF DepNum DepName MngrID A001 Marketing 5 A001 Marketing 6 A002 Sales 2 A002 Sales 5 A003 Consulting 6 A003 Consulting 10 A001 A002 A003 A001 A002 A period of validity current time SELECT * FROM Department FOR SYSTEM_TIME AS OF ' '

15 Department (current + history) BETWEEN..AND DepNum DepName MngrID A001 Marketing 5 A001 Marketing 6 A002 Sales 2 A002 Sales 5 A003 Consulting 6 A003 Consulting 10 A001 A002 A003 A001 A002 A period of validity current time SELECT * FROM Department FOR SYSTEM_TIME BETWEEN ' ' AND ' '

16 Department (current + history) CONTAINED IN DepNum DepName MngrID A001 Marketing 5 A001 Marketing 6 A002 Sales 2 A002 Sales 5 A003 Consulting 6 A001 A002 A003 A001 A002 A003 Consulting 10 A period of validity current time SELECT * FROM Department FOR SYSTEM_TIME CONTAINED IN (' ', ' ')

17 Department (current + history) ALL DepNum DepName MngrID A001 Marketing 5 A001 Marketing 6 A002 Sales 2 A002 Sales 5 A003 Consulting 6 A003 Consulting 10 A001 A002 A003 A001 A002 A period of validity current time SELECT * FROM Department FOR SYSTEM_TIME ALL

18 DEMO

19

20 Expression AS OF <date_time> FROM <start_date_time> TO <end_date_time> BETWEEN <start_date_time> AND <end_date_time> CONTAINED IN (<start_date_time>, <end_date_time>) ALL Qualifying Rows SysStartTime <= date_time AND SysEndTime > date_time SysStartTime < end_date_time AND SysEndTime > start_date_time SysStartTime <= end_date_time AND SysEndTime > start_date_time SysStartTime >= start_date_time AND SysEndTime <= end_date_time All rows

21 Keep, keep, keep!???!#$^* Cut, cut, cut! Business owners, end-users, DBA Storage admins, budget owners,

22 SELECT * FROM Department FOR SYSTEM_TIME AS OF ' ' Solution: History as a stretch table: PeriodEnd < Now - 6 months Azure SQL Facts: Database 1. History is much bigger than actual data 2. Retained between 3 and 10 years 3. Warm : up to a few weeks/months 4. Cold : rarely queried

23 Stretch Database

24 Stretch Database Mike Wan ox7ff654ae6d 3/18/2005 Microsoft Azure Effettuare stretch delle tabelle che contengono cold data con sicurezza da scenari on-prem verso Azure grazie al remote query processing economico Order history Stretch to cloud Name SSN Date Order Steve Karimi history Customer data cm61ba906fd 2/28/2005 Name Mike Wan SSN ox7ff654ae6d Product data 3/18/2005 Date Steve Sapna Karimi Matuszyk cm61ba906fd i2y36cg776rg 2/28/2005 4/10/2005 Order History Mike SQL Mojoe Wan ox7ff654ae6d nx290pldo90l 3/18/2005 4/27/2005 Sapna Matuszyk i2y36cg776rg 4/10/2005 Query App L intera tabella resta online e rimane accessibile dale applicazioni esistenti Nessuna modifica alle applicazioni esistenti Sicuro by default con la possibilità di aggiungere ulteriore sicurezza con Always Encrypted e Row Level Security SQL Mojoe nx290pldo90l 4/27/2005 Blazej Zwilling ypo85ba616rj 5/12/2005 Cheng Jeswani bns51ra806fd 5/22/2005 Nikhat Lindell mci12hh906fj 6/07/2005 Whitney Lang utb76b916gi 6/18/2014 Lorenzo Olds px61hi9306fj 7/1/2014 Sophie Cook ol43bi506gd 7/12/2014 Aida Durham tx83hal916fi 7/29/2014 Jaclyn Wade nb95re926gi 8/10/2014

25 Table Partitioning 1. SWITCH OUT 2. MERGE RANGE 3. SPLIT RANGE

26 Custom Delete Script 1. Turn off System- Versioning 2. Delete old rows * 3. Turn on System- Versioning * Limitare il numero di righe per ogni iterazione per evitare lock prolungati sulla tabella

27 Builtin retention policy Enabled at the database level. SELECT is_temporal_history_retention_enabled, name FROM sys.databases ON by default user can change it with ALTER DATABASE statement ALTER DATABASE <mydb> SET TEMPORAL_HISTORY_RETENTION ON Automatically set to OFF after point in time restore operation

28 Builtin retention policy CREATE TABLE dbo.websiteuserinfo ( [UserID] int NOT NULL PRIMARY KEY CLUSTERED, [UserName] nvarchar(100) NOT NULL, [PagesVisited] int NOT NULL, [ValidFrom] datetime2 (0) GENERATED ALWAYS AS ROW START, [ValidTo] datetime2 (0) GENERATED ALWAYS AS ROW END, PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo) ) WITH ( SYSTEM_VERSIONING = ON ( HISTORY_TABLE = dbo.websiteuserinfohistory, HISTORY_RETENTION_PERIOD = 6 MONTHS ) );

29 Builtin retention policy Possibili valori per HISTORY_RETENTION_PERIOD: - DAYS - WEEKS - MONTHS - YEARS - INFINITE (default se non specificato) Impostando SYSTEM_VERSIONING = OFF viene perso il valore dell impostazione HISTORY_RETENTION_PERIOD ALTER TABLE dbo.websiteuserinfo SET (SYSTEM_VERSIONING = ON (HISTORY_RETENTION_PERIOD = 9 MONTHS));

30 Builtin retention policy ALTER TABLE [dbo].[inventory] SET (SYSTEM_VERSIONING = ON (HISTORY_RETENTION_PERIOD = 6 MONTHS)); GO

31 Builtin retention policy IF is_temporal_history_retention_enabled = ON THEN The query plan includes additional filter applied to end of period column (ValidTo) in the Clustered Index Scan operator on the history table so, do not rely your business logic on reading history table beyond retention period as you may get inconsistent or unexpected results

32 - A temporal table must have a primary key defined - History table cannot have constraints; primary key, foreign key, table or column constraints - INSERT and UPDATE statements cannot reference the SYSTEM_TIME period columns - TRUNCATE TABLE is not supported while SYSTEM_VERSIONING is ON - Direct modification of the data in a history table is not permitted - INSTEAD OF triggers not permitted on current and history table, AFTER triggers permitted only on CURRENT TABLE - REPLICATION usage is limited, some objects/properties are not replicated - * ON DELETE CASCADE and ON UPDATE CASCADE are not permitted on the current table. * NOTE: This limitation applies to SQL Server 2016 only. CASCADE options are supported in SQL Database and SQL Server 2017 starting from CTP 2.0.

33 period columns in a temporal table hold the time in the UTC time zone as DATETIME2 values

34

35

36

37 Q&A Questions?

38 THANKS! #SQLSatCatania #sqlsat728