SQL Server 2014 Highlights der wichtigsten Neuerungen In-Memory OLTP (Hekaton)

Transcription

1 SQL Server 2014 Highlights der wichtigsten Neuerungen Karl-Heinz Sütterlin Meinrad Weiss March 2014 BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA 1

2 SQL Server

3 Driver for In Memory OLTP (HEKATON) Intel CPU trends and Memory prices($/gb) Computing power holds Moore Law (due to parallelism) CPU clock frequency stalled Memory has gotten a LOT cheaper 3

4 In-Memory OLTP gains No improvement Example 1 Connectivity 2 ms 1X 2 ms Execution and data access 1 ms 2 ms 5 ms 30X Logging 1X Total 20% 0.03 ms 2 ms 4 ms 20-30x improvement Same latency Less volume Example 2 Connectivity 0.5 ms 1X 0.5 ms Execution and data access 10 ms 0.2 ms 10.7 ms 10X Logging 1X Total 600% 1ms 0.2 ms 1.7 ms 4

5 In-Memory OLTP gains Disk-based tables Page Latch Memory-optimized tables 5

6 Drivers Hekaton Tech Pillars Customer Benefits Hekaton Architecture Pillars High performance data operations Efficient, business-logic processing Frictionless scale-up Hybrid engine and integrated experience Main-Memory Optimized T-SQL Compiled to Machine Code High Concurrency SQL Server Integration Optimized for in-memory data Indexes (hash and range) exist only in memory No buffer pool, B-trees Stream-based storage T-SQL compiled to machine code via C code generator and VC Invoking a procedure is just a DLL entry-point Aggressive compile-time Multi-version optimistic concurrency control with full ACID support Core engine uses lock-free algorithms No lock manager, latches or spinlocks Same manageability, administration & development experience Integrated queries & transactions Integrated HA and backup/restore Hardware trends Business Steadily declining memory price, NVRAM Stalling CPU clock rate Many-core processors TCO 6

7 Hekaton Integration and Application Migration Client App TDS Handler and Session Management Natively Compiled SPs and Schema Hekaton Compiler Parser, Catalog, Optimizer T-SQL Query Execution Existing SQL Component Hekaton Component Query Interop T1 T2 T3 Tables Generated.dll Indexes Memory Optimized Tables & Indexes SQL Server.exe Buffer Pool for Tables & Indexes T1 T2 T3 T1 T2 T3 Memory Optimized 2013 Table Trivadis Filegroup Transaction Log Data Filegroup

8 Create Table DDL CREATE TABLE [Customer]( [CustomerID] INT NOT NULL Hash Index PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT = ), [Name] NVARCHAR(250) NOT NULL INDEX [IName] HASH WITH (BUCKET_COUNT = ), ) [CustomerSince] DATETIME NULL WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA); Secondary Indexes are specified inline 8 This table is memory optimized This table is durable

9 Memory Optimized Table Creation CREATE TABLE DDL Code generation and compilation Table DLL produced Table DLL loaded 9

10 Memory Optimized Tables and Indexes Timestamps Chain ptrs Name City Hash index on Name 200, John Beijing Hash index on City 100, 200 John Paris 90,150 Susan Bogota Garbage Collection Removes Unused Rows 50, Jane Prague 10

11 Limitations in Initial Release Optimized for high-throughput OLTP No DML triggers No XML and no CLR data types Optimized for in-memory Rows are at most 8060 bytes no off row data No Large Object (LOB) types like varchar(max) Scoping limitations No FOREIGN KEY and no CHECK constraints No IDENTITY No schema changes (ALTER TABLE) need to drop/recreate table No add/remove index need to drop/recreate table 11

12 Accessing Memory Optimized Tables Interpreted T-SQL Access (InterOP) Access both memory- and diskbased tables Less performant Virtually full T-SQL surface When to use Ad hoc queries Reporting-style queries Speeding up app migration Eliminating Latch Contention Natively Compiled Procs Access only memory optimized tables Maximum performance Limited T-SQL surface area When to use OLTP-style operations Optimize performance critical business logic 12

13 Create Procedure DDL CREATE PROCEDURE DATETIME WITH AS NATIVE_COMPILATION, SCHEMABINDING, EXECUTE AS OWNER BEGIN ATOMIC WITH (TRANSACTION ISOLATION LEVEL = SNAPSHOT, LANGUAGE = 'us_english') -- insert T-SQL here 13 END Session settings are fixed at create time This proc is natively compiled Native procs must be schema-bound Execution context is required Atomic blocks Create a transaction if there is none Otherwise, create a savepoint

14 Procedure Creation CREATE PROC DDL Query optimization Code generation and compilation Procedure DLL produced Procedure DLL loaded 14

15 Suitable Application Characteristics Application is suited for in-memory processing All performance critical data already fits in memory Transaction locking or physical latching causing stalls and blocking Application is OLTP-Like Relatively short-lived transactions High degree of concurrent transactions from many connections Examples: Stock trading, travel reservations, order processing Application porting simplified if Stored procedures used Performance problems isolated to subset of tables and SPs 15

16 High data input or Shock Absorber Shock Absorber Handles constant high volumes Atypical spikes in workload Usually latching bottleneck on input Push to disk-based: Hot and cold data Memory footprint Only certain data fits into memory Desire compression on colder data Workload pattern Mixed OLTP Reporting/BI query workloads Columnstore indexing Data Loading Table1_MemoryOptimized Table2_Diskbased Optional data movement 16

17 Staging tables and temp tables Goal: Throughput Bottlenecks: Typically IO (or latching) Staging table containing transient data SCHEMA_ONLY in many cases Ability to replay data from source if data is lost Reduce IO impact Memory optimized-tables can be used as temp tables Data Loading - ETL Native Compiled - Transformation Staging SCHEMA_ONLY Table2_DataWarehouse 17

18 Limits No foreign key or check constraints No T-SQL function calls in compiled stored procedures Transact-SQL Constructs Not Supported by In-Memory OLTP 18

19 Application patterns to avoid with In-Memory OLTP Where you lack the ability to change code Hardware limitations Non-OLTP workloads: Full data warehouse Long running reporting Heavy parallel queries XML/full text Bottleneck cannot be addressed by migration (outside of In-memory OLTP engine) 19

20 Key takeaways In-Memory OLTP engine Provides a solution for applications with common RDBMS performance and scale bottlenecks Meets demanding scale and performance goals with a relational database Common design patterns In-Memory OLTP High-performance OLTP High data input or Shock Absorber Relational cache Read-scale ETL target and staging/temp tables To assess workload for In-Memory OLTP: Understand your bottleneck (AMR can help) Migrate only necessary pieces of your solution Execute testing at scale to understand the full benefits you may achieve Some applications may require a lot of work to achieve the gains, while others migrate quite easily Some applications may not be a good fit 20