Field Testing Buffer Pool Extension and In-Memory OLTP Features in SQL Server 2014

Size: px

Start display at page:

Download "Field Testing Buffer Pool Extension and In-Memory OLTP Features in SQL Server 2014"

Lindsay Cook
5 years ago
Views:

1 Field Testing Buffer Pool Extension and In-Memory OLTP Features in SQL Server 2014 Rick Heiges, SQL MVP Sr Solutions Architect Scalability Experts Ross LoForte - SQL Technology Architect - Microsoft

2 Changing World Rick Heiges

3 Things Have Changed in 10 Years! (So Has SQL Server!) Typical OS Win 2003 R2 Win 2012 R2 SQL Version SQL 2000 SQL 2014 CPU Single-Core CPU (some HT) Multi-Core CPU RAM 4GB 8GB (up to 64GB) 32GB + (up to 2TB) Storage DASD (some SAN) SAN and Flash Monitor Desktop Monster CRT Multi-Display Flat Panel Large DB 300GB was Large 1TB DBs are not uncommon Virtualization Novelty (Non-Prod Only) Pervasive High Availability Failover Clustering AlwaysOn AG and FCI

4 Field Testing Background Location: Fusion-IO Lab in SLC Players: Fusion-IO Brian Walters Joel Grace Microsoft MTC Chicago Lead Ross LoForte Program Manager for BPE Evgeny Krivosheev Program Manager for In-Memory Jamie Reding Scalability Experts Rick Heiges, et. al.

5 Goals Explore New Features of SQL Server where having the right storage solution makes a big difference Extend the official testing to new levels to find scalability limits Determine Use Cases for Further Study Whitepapers Documenting Field Testing

6 Field Testing the Buffer Pool Extension Feature

7 Extending the Buffer Pool Can extend the Buffer Pool to non-volatile storage (SSD or SSD Array) Only Clean pages are written to BPE in order to minimize potential data loss Potential Performance Gains Especially for Read- Intensive OLTP workloads No Application Changes. Enable the Feature and go Simple Syntax

8 Using BPE Enable Server Configuration Syntax Example ALTER SERVER CONFIGURATION SET BUFFER POOL EXTENSION ON (FILENAME= F:\BufferPool\FusionIO.bpe', SIZE = 125 GB); Check Status SELECT * FROM sys.dm_os_buffer_pool_extension_configuration Turn off BPE ALTER SERVER CONFIGURATION SET BUFFER POOL EXTENSION OFF;

9 BPE Good Scenarios and Otherwise Sweet Spot Read Heavy OLTP workloads Indexes in BPE High Throughput SSD Storage (High Endurance SSDs suggested) Limitations Data Warehouse workloads Write Heavy OLTP workloads Now on Standard Edition in 2014!!!

10 Demo BPE

11 Interpreting the Buffer Pool Extension Testing Results

12 Test Configuration Dell PowerEdge R720 2 Intel 2.7GHz 12 core Ivy Bridge Processors (Total of 24 cores, HT on) 384GB RAM 2.4TB iodrive2 Duo Enterprise Class Storage Array Data Storage Pool: 12 15k SAS spindles Log Storage Pool: 12 15k SAS spindles Total of 3 LUNs: 1 for Data, 1 for TempDB, 1 for Logs

13 16GB RAM vs. 8GB RAM + 256GB BPE

14 Database Cache Database cache immediately extends beyond the allocated 8GB of DRAM by 2700%.

15 Process More Data Batch Requests increase by 37% and CPU % increases by 300%

16 More Reads from BPE BPE becomes the primary sources for logical reads

17 Increase Throughput 1GB/s is average

18 Consistent Low Latency is Key Fusion-io provides a near DRAM-like experience

19 Offload Reads Disk array life extender

20 Increased Transaction Log Write Performance Offloading reads increases write performance

21 Interpreting the Results High speed flash drives delivers "DRAM like" performance: Increasing CPU utilization 300%. Increasing transaction log throughput by 20% Offloading database reads from the disk array by 20% Sweet Spot appears to be systems with approx. 64GB RAM.

22 Use Cases Reporting on OLTP systems Consolidation inside VMs for underutilized CPUs

23 Field Testing the In- Memory OLTP Feature Ross LoForte

24 Response Time What is SQL Server In-Memory OLTP? Hard drives 10 years This is not DBCC PINTABLE from a previous version.

25 Durability Durable or non-durable Default is durable Non-durable tables are useful for transient data Persisted in a single memory-optimized filegroup Different access pattern (Sequential IO) vs. disk tables Recommend SSD or fast (15K e.g.) SAS drives Filegroup can have multiple containers (volumes) Additional containers aid in parallel recovery; recovery happens at the speed of IO Committed transactions are logged A common system bottleneck; SSD recommended

26 Integrated Experience Backup and restore Full and log backup and restore is supported Piecemeal restore is supported Failover clustering Failover time depends on size of durable memory-optimized tables Availability Groups Secondary has memory-optimized tables in memory Failover time is not dependent on size of durable memory-optimized tables DMVs, catalog views, performance monitor counters, XEvents Monitoring memory, garbage collection activity, and transaction details SQL Server Management Studio (SSMS) Creating, managing, and monitoring tables; databases and server

27 Performance Factors for In-Memory OLTP Data resides in memory at all times Must configure SQL server with sufficient memory to store memory-optimized tables Failure to allocate memory will fail transactional workload at run-time Integrated with Resource Governor Recommend using dedicated resource pool Bind a database to a resource pool Disk Subsystem performance CPU performance

recommendations Memory Optimization Advisor Guides the process of migrating a table to a memory-optimized

28 Migration Analysis, Migrate, and Report (AMR) tool Transaction performance collector Collects existing tables data and stored procedures to analyze workloads Transaction performance analysis reports for recommendations Memory Optimization Advisor Guides the process of migrating a table to a memory-optimized table Native Compilation Advisor Evaluates the process of migrating a stored procedure to a natively compiled stored procedure

29 Demo In-Memory OLTP Ross LoForte

30 Suitable Application Characteristics Suited for in-memory processing Critical data will fit in memory Transaction locking or physical latching OLTP-Like Relatively short-lived transactions High degree of concurrent transactions from many connections Examples: Stock trading, travel reservations, order processing Porting simplified Stored procedures used Performance problems isolated to subset of tables and SPs

31 Tables Limitations in SQL Server 2014 Triggers: no DDL/DML triggers Data types: no datetimeoffset, no LOBs, no XML and no CLR data types Constraints: no FOREIGN KEY and no CHECK constraints No schema changes (ALTER TABLE) need to drop/recreate table No add/remove index need to drop/recreate table Natively Compiled Stored Procedures Supported: SELECT, INSERT, UPDATE, DELETE, IF, WHILE, TRY CATCH INNER JOIN, GROUP BY, AND Table-Valued Parameters Not supported: OUTER JOIN, DISTINCT, OR, Subqueries, CASE Limited built-in functions [core math, date/time, and string functions are supported]

32 Examining the In-Memory OLTP Testing Results

33 Interpreting the Results Business Value Key Performance Indicators Enterprise- Class Disk Array with SQL Server 2014 In- Memory OLTP iodrive2 Duo 2.4TB with SQL Server 2014 In- Memory OLTP Business Process Improvement Improve Customer Experience User Transaction Wait Time (µ) Reduced user wait times by 91% Serve More Customers Transaction Throughput (MB/s) Increased throughput by 409% Improve Business Productivity Total Transactions Processed (over 45 minutes) 6,362,883 28,328,639 Processed 445% more transactions Deliver on Internal Service Level Agreements Database Startup Time (sec) Reduced startup time by 67%

34 Test Configuration Dell PowerEdge R720 2 Intel 2.7GHz 12 core Ivy Bridge Processors (Total of 24 cores, HT on) 384GB RAM 2.4TB iodrive2 Duo Enterprise Class Storage Array Data Storage Pool: 12 15k SAS spindles Log Storage Pool: 12 15k SAS spindles Total of 3 LUNs: 1 for Data, 1 for TempDB, 1 for Logs

35 Overall Performance Overall TPCC Performance Comparison SQL Server 2014 In-Memory OLTP iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 2, ,329 1, Transaction Log Write Latency (microseconds) 640 Transaction Log Read Latency (microseconds) Transaction Log Write Megabytes/sec Transaction Log Read Megabytes/sec 217 Memory-Optimized Filegroup Write Latency (microseconds) Memory-Optimized Filegroup Write Megabytes/sec iodrive2 Duo 2.4TB Enterprise-Class Disk Array Flash products increase In-Memory OLTP performance more effectively than an enterprise disk array.

36 Transaction Log Performance SQL Server 2014 In-Memory OLTP Transactions/sec iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array 52, x More Transactions 11,810 0 iodrive2 Duo 2.4TB Enterprise-Class Disk Array iomemory system enables 4.4X more user transactions per second

37 Transaction Logs Write Megabytes/sec 4X faster transaction log writes eliminates I/O slowdowns.

38 Transaction Logs Write Latency (usec) 90%+ lower latency delivers consistently fast response times, which improves the user experience and helps IT confidently meet SLAs.

39 Megabytes Database Startup Tests SQL Server 2014 In-Memory OLTP Loading 120GB In-Memory Data Read Megabytes/sec iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array iodrive2 Duo 2.4TB Enterprise-Class Disk Array Database startup on iomemory is 308% faster than from an enterprise disk array.

40 Memory Optimized File Group Performance SQL Server 2014 In-Memory OLTP Memory-Optmized Filegroup Write Latency (microseconds) iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array iodrive2 Duo 2.4TB Enterprise-Class Disk Array iomemory products enable 84% faster in-memory checkpoint processes with much less variatio

41 File Group Performance - Checkpoints SQL Server 2014 In-Memory OLTP Memory-Optmized Filegroup Write Megabytes/sec iodrive2 Duo 2.4TB vs. Enterprise-Class Disk Array iodrive2 Duo 2.4TB Enterprise-Class Disk Array Continuous checkpoints are faster due to 75% higher average bandwidth

42 Scaling Beyond Users (24K Users) SQL Server In-Memory OLTP iodrive2 Duo 2.4TB Memory-Optimized File Group and Transaction Log Performance Average of Transaction Log Write Latency (microseconds) Average of Transaction Log Write Megabytes/sec Average of Memory-Optimized Filegroup Write Latency (microseconds) Average of Memory-Optimized Filegroup Write Megabytes/sec With 24,000 users write latency consistently stays under 220 microseconds

43 Staying Consistent at 24,000 Users SQL Server In-Memory OLTP iodrive2 Duo 2.4TB Transactions/sec Time Series (45 minutes) Database transactions average 109,000 per second once all 24,000 users are on the system

44 Use Cases Primary Eliminate Latch/Lock contention in systems where Resource Contention can bottleneck performance Other In-Memory inside a VM High Performance while maintaining Virtualization benefits Startup Performance

45 Summary Rick Heiges

46 Summary Choosing the right type of storage can dramatically affect performance of Key SQL Server 2014 Features BPE appears to be linear in nature in early testing (constraints in time and workload prevented going further) In-Memory OLTP benefits from wicked fast local storage for streamed logs and hardened data on startup

47 References In-Memory Whitepaper: < in-memory-db-with-iomemory/> In-Memory Datasheet: < Blog post: < Fusion s press release: < Fusion s SQL 2014 landing page: <

48 Questions?

49 Thank You for Attending

Accelerating Microsoft SQL Server Performance With NVDIMM-N on Dell EMC PowerEdge R740

Accelerating Microsoft SQL Server Performance With NVDIMM-N on Dell EMC PowerEdge R740 A performance study with NVDIMM-N Dell EMC Engineering September 2017 A Dell EMC document category Revisions Date