SQL Server 2014 In-Memory OLTP: Prepare for Migration. George Li, Program Manager, Microsoft

Transcription

1 SQL Server 2014 In-Memory OLTP: Prepare for Migration George Li, Program Manager, Microsoft

2 Drivers Architectural Pillars Customer Benefits In-Memory OLTP Recap High performance data operations Efficient business-logic processing Frictionless scale-up Hybrid engine and integrated experience Main-Memory Optimized T-SQL Compiled to Machine Code High Concurrency SQL Server Integration Optimized for inmemory data Indexes (hash and range) exist only in memory No buffer pool, B-trees Stream-based storage T-SQL compiled to machine code via C code generator and VC Invoking a procedure is just a DLL entry-point Aggressive compile-time Multi-version optimistic concurrency control with full ACID support Core engine uses lockfree algorithms No lock manager, latches or spinlocks Same manageability, administration & development experience Integrated queries & transactions Integrated HA and backup/restore Hardware trends Business Steadily declining memory price, NVRAM Stalling CPU clock rate Many-core processors TCO 2

3 Migration Steps Analyze application Is the application suitable for In-memory OLTP? Define goals and measure Identify bottlenecks Analysis part of AMR tool can help Understand considerations Memory, Storage, Hardware, Design Staged migration Migrate part of AMR tool can help Measure against goals 3

4 Analyze Application In-Memory OLTP is a good fit for Performance-Critical OLTP (think order processing or trading High Data-Input Rate (nicknamed Shock Absorber ) In-Memory OLTP as ETL staging table Relational Cache (think Session State) In-Memory OLTP is not a good fit for No permission for code changes Full data warehousing Long-running reporting workload (try Columnstore) Use a lot of XML manipulation/full-text searches Heavily parallelized query 4

5 Establish Performance Baseline Production is better! More accurate and realistic data But hard to get access Simulation works, but Scale, transaction mix, profiling period matters Do: Try to be as realistic as possible with scale, transaction mix, hardware configuration, etc. Don t: Test on a single workstation with inappropriate configuration Profiling Tools Ostress, Distributed Replay, Visual Studio Test System In-Memory OLTP s Analysis, Migrate and Report Toolset Performance Monitor 5

6 Scale Matters ostress.exe -S. E -dadventureworks2012 -Q"EXEC = n<varies> 6

7 Identify Bottlenecks Table contention Do: Look at dm_db_index_operational_stats, dm_os_latch_stats, and Latch Waits/sec counter Do: Target table with heaviest latching Table access Do: Look at dm_db_index_operational_stats Do: Target table with heaviest accesses Stored procedure CPU usage Do: Look at dm_exec_query_stats Do: Target stored procedure with highest CPU usage Tooling in place In-Memory OLTP s Analysis, Migrate and Report Toolset 7

8 Demo Analyzing a Workload 8

9 Durability Considerations Durable Operations on data are always fully logged Data is stored in checkpoint files on disk and can be recovered Requires storage considerations Non-Durable Transient data with no log or checkpoint files Schema can be recovered but data cannot be recovered after server restart or failure Reduces dependency on disk drives, improves overall transaction throughput Not universally applicable use for select scenarios only 9

10 Storage Considerations for Durable Tables Data Drive Each database can have maximum of 8192 checkpoint files Do: Use workload s steady state to determine disk usage Do: Have checkpoint files on drive with high sequential I/O Do: Spread checkpoint files over odd numbers of containers Log Drive Index operations and UNDO records are not logged Do: Have log on drive with low latency 10

11 Index Considerations Nonclustered Index Supports all operations but mainly use for range scans and ordered scans Performs better with inequality predicate No bucket-count configuration needed but uni-directional General purpose index Hash Index Use to optimize for point look-ups, other queries will result in full scans Performs better with equality predicate that covers all key columns Fixed bucket-count configuration, cannot be altered Good for unique values over known maximum number of rows Too many duplicate keys will cause excessive chaining and reduce performance 11

12 Index Considerations Choosing Bucket Count Applies to Hash Index only Do: Choose at least as many buckets as the key space (automatically rounded up to next power of 2) Performance impacted by bucket count too low or too high, too many recurring values SELECT without a where clause will result in scanning of all buckets Choosing Index Key Columns Applies to both indexes, though more important for Hash Index Can create both type of indexes covering the same set of key columns Do: Carefully examine queries that use the index Hash Index: Choose only the columns that are used in the equality predicate, extra columns harmful Nonclustered Index: Choose index direction that conforms to seek direction 12

13 Memory Considerations Data Consumption Maximum of 8192 bytes per row, 8060 used for row content Padding and overheads can happen depending on column data type Row versions stay in memory until transactions with corresponding end-timestamp are completed Garbage collection, unless under memory pressure, is asynchronous and can take time to remove rows Index Consumption Indexes contain pointers and not actual rows Hash Index: fixed, function of bucket count, 8 bytes per bucket Nonclustered Index: dynamic, function of row count, overhead Overall In-Memory OLTP Consumption Do: Use Resource Governor to control memory allocated 13

14 Estimating Memory Consumption Memory Size = Table Size + SUM(Index Size) Table Size = Row Size * Row Count Row Size = SUM(Column Sizes) + Row Header Size Row Header Size = * Index Count Column Size = Dependent on column type and associated padding/overhead Hash Index Size = Bucket_Count * 8 bytes Nonclustered Index Size= Row Size * (Key Size + 8) bytes Provision ~2x Memory Size due to row versioning overhead Details available in Books Online 14

15 Interpreted vs. Native When to use interpreted mode? CPU is not a bottleneck Latency requirements are being met Need access to a lot more surface area than native compilation provides When to use native compilation? CPU is a bottleneck Latency requirements are not being met Supported surface areas are sufficient Documented workarounds are acceptable

16 Limitations and Workarounds Tables Unsupported: SPARSE column, DBCC CHECKTABLE, etc. Well-defined workarounds exist for Constraints, Foreign Keys, Triggers, etc. Involved workarounds exist for SQL_VARIANT, LOB column, etc. Indexes Unsupported: Filtered index, nullable index key column, etc. Involved workarounds exist for collation, etc Stored Procedures Unsupported: ROLLUP, EXTERNAL, non-constant parameter defaults, etc. Well-defined workarounds exist for SELECT INTO, CASE, GOTO, etc. Involved workarounds exist for cursors, table-valued functions, UNION, OR, NOT, etc. Details available in Books Online

17 Measure Against Goals Measure using the same tools for profiling Important to compare against the baseline with the same parameters Might notice CPU usage increase May be a good thing: more work done on the CPU Migration is an iterative process Other bottlenecks may emerge as some are eliminated More iterations, more effort, more gains Must carefully evaluate cost vs. benefit

18 Demo: Migrating a Table to a Memory-Optimized Table 18

19 Recap: Migration Steps Analyze application Is application suitable for In-memory OLTP? Define goals and measure Identify bottlenecks Analysis part of AMR tool can help Understand considerations Memory, Storage, Hardware, Design Staged migration Migrate part of AMR tool can help Measure against goals

20 Questions?

21 Thank You for Attending