Persistence Is Futile- Implementing Delayed Durability in SQL Server

Transcription

1 Mark Broadbent #SqlSat675 Persistence Is Futile- Implementing Delayed Durability in SQL Server

2 Sponsor #SqlSat675 18/11/2017

3 Organizzatori GetLatestVersion.it #SqlSat675 18/11/2017

4 Agenda We will also need to discuss and explain In-Memory OLTP in this context! 1 Transactions & Breaking ACID Properties 2 Architecture 3 Delayed Durability Implementation Limitations 4 The best part of the presentation Gin O Clock 5

5 A C I D

6 Transactions can be Explicit Auto-Commit INSERT DELETE BEGIN DISTRIBUTED TRAN COMMIT TRAN MSDTC Distributed MSDTC Batch Scope (MARS) UPDATE INSERT DELETE GO BEGIN TRAN COMMIT TRAN INSERT DELETE COMMIT TRAN BEGIN TRAN spid 115 BEGIN TRAN UPDATE Binding Token BEGIN TRAN INSERT spid 162 Implicit (evil muthas!) Session Binding

7 Transactions Transaction Name Transaction Mark BEGIN TRAN transaction_1 BEGIN TRAN --do something COMMIT TRAN SAVE TRAN BEGIN TRAN --do something else COMMIT COMMIT savepoint WITH MARK restorepoint Savepoint Nested Transaction IF {something_wrong} THEN ROLLBACK TRAN savepoint Outer Transaction *1 *1 Not to be confused with Atomic Blocks (we ll discuss these later!)

8 Demo Transactions and broken ACID properties

9 Durable Transactions flushes: On commit On full On Checkpoint *1 Writer async Buffer 1 Signal to flush Control returned to spid on log flush VLF header BEGIN TRAN COMMIT file (.ldf) file Header Version FseqNo Size *1 Data pages that need to be written to disk will require Buffers containing earlier LSNs to be flushed to disk Block Header Record 1 Record 2 Record n Slot Array 512 bytes to 60KB Virtual Files (VLFs) Inactive VLFS Record has unique LSN LSN has 3 part name consisting of: VLF Number + Block Offset + Slot Number growth size <=64MB then 4 VLFs > 64MB AND <= 1GB 8 VLFs > 1GB 16 VLFS Think about how big your log needs to be from the beginning

10 Delayed Durable Transactions flushes: On full On Checkpoint *1 sp_flush_log Writer async Buffer 1 Control returned to spid immediately VLF header BEGIN TRAN COMMIT file (.ldf) file Header Version FseqNo Size *1 Data pages that need to be written to disk will require Buffers containing earlier LSNs to be flushed to disk Block Header Record 1 Record 2 Record n Slot Array 512 bytes to 60KB Virtual Files (VLFs) Inactive VLFS Record has unique LSN LSN has 3 part name consisting of: VLF Number + Block Offset + Slot Number growth size <=64MB then 4 VLFs > 64MB AND <= 1GB 8 VLFs > 1GB 16 VLFS Think about how big your log needs to be from the beginning

11 Delayed Durability Hierachy TRANSACTION (On-disk or IM tables) Except If CDC is enabled Is a cross database transaction Is a distributed transaction DATABASE DISABLED ALLOWED FORCED OFF ON ATOMIC BLOCK (IM tables via Native Compiled Stored Procs) but delayed durability not guaranteed regardless! OFF ON

12 Demo On disk delayed durability

13 In-Memory OLTP Overview New concurrency model (as of SQL Server 2014) Provides lockless and latchless operation All row and index data in memory (optimised for memory data structures) All versioning In-Memory Physical persistence (SCHEMA_AND_DATA) Hash Index/es BW Tree Index/es In-Memory constructs Transaction Physical persistence Checkpoint file pair/s

14 In-Memory OLTP ging and Improvements Persistence (and recovery) through Transaction and checkpoint file pair/s but log file is the primary source! Improvements block compression No undo record generation Indexes not persisted, rebuilt on start-up so NO index maintenance logging file (.ldf) Writer async Buffer 1 Signal to flush Buffer 2 file Checkpoint threads but on disk structures become an even bigger potential bottleneck a b c d e f g h IMOLTP Datafile 1 (container) IMOLTP Datafile 2 (container)

15 Demo In-Memory delayed durability

16 Waits and Performance Monitoring LOGBUFFER wait time taken to create log record in log buffer WRITELOG wait time taken for log buffer to be flushed to logfile Use sys.dm_io_virtual_file_stats to look at file IO, sizes and stalls Performance Monitors e.g. flushes per second counter of Databases counter set Transactions per second counter of Databases counter set

17 Limitations and Special Cases Ignored when CDC enabled tables Cross-database or distributed transactions Not supported in Transaction Replication Data is delayed (temporarily missing) in Sync Read-only replicas High Safety Database Mirrors backups and shipping targets Warning! Data loss is very possible in these scenarios! *1 Basically crash recovery in standalone and automatic failovers in HADR configurations can and possibly will result in data loss!

18 Summary Delayed Durability attempts to make log flushes more efficient A compromise between Improving performance over durability Can force durability on every database transaction, specific transactions or atomic block via Natively Compiled Stored Procedures. In-Memory OLTP could remove all other bottlenecks so Delayed Durability *could* remove log buffer waits However IMOLTP already provides some great logging improvements You can lose data, even under highly available environments!

19 Thank you for listening! Twitter: retracement Blog: Slideshare: Demo: