AMA: Open Questions and Answers on SQL Server in Azure VMs. Silvano Coriani, Program Manager AzureCAT Microsoft

Size: px

Start display at page:

Download "AMA: Open Questions and Answers on SQL Server in Azure VMs. Silvano Coriani, Program Manager AzureCAT Microsoft"

Millicent George
6 years ago
Views:

1 AMA: Open Questions and Answers on SQL Server in Azure VMs Silvano Coriani, Program Manager AzureCAT Microsoft

3 Agenda Some of the questions people always ask me - General performance related questions - What VM size should I use? - I ve a XXX-GB database, can I migrate it to Azure? - How do I troubleshoot my SQL Server in Azure? - Storage - What VM size should I use? - How Azure Data Disks work? - How should I organize my storage layout? - HA/DR - What options do I have? - Can I leverage AlwaysOn for my production databases? - Can i create AGs beween on premises and Azure (hybrid)? - Can I create cross-region AGs? 3

4 General performance related questions 4

5 I ve a XXX-GB db, can I move it to Azure? Active dataset size? How large is the active portion of your database? Can be mainly hosted in memory? Log throughput? Is it a write intensive database? Can I leverage minimal logging for data loading? What log generation rate/throughput do I need? Latency dependent? Is this an interactive workload? How chatty my application is? Can I batch some database calls instead? 5

What VM size should I use? VM Size CPU Cores Memory Max. IOPS # Data Disks Temp Drive A0 Shared 768 MB 1x500 1 20 GB A1 1 1.75 GB 2x500 2 70 GB A2 2 3.

6 What VM size should I use? VM Size CPU Cores Memory Max. IOPS # Data Disks Temp Drive A0 Shared 768 MB 1x GB A GB 2x GB A GB 4x GB A3 4 7 GB 8x GB A GB 16x GB A GB 4x GB A GB 8x GB A7 8 (AMD Opteron GHz) 56 GB 16x GB

7 What VM size should I use? A3+ VMs give you the right combination of compute, memory and IO capacity (# of data disks) to run production workloads A8-A9 have faster Intel Xeon E processors that provide not only more compute power, but also better IO performance (increased disk bandwidth within the 500 IOPs per disk boundaries) Memory intensive instances can help accommodating more data in memory and reduce IO latency issues 7

8 Troubleshooting process Perfmon analysis Key performance counters Powershell to automate Network diagnostics SQLIO testing and parsing SQLIO command line options Using DMVs and Xevents to understand SQL Server performance sys.dm_os_wait_stats sys.dm_exec_query_stats sys.dm_io_virtual_file_stats

9 Key Performance Counters Logical disk counter Typical storage term Suggested actions in Windows Azure virtual machine environment Disk reads / second Disk writes / second IOPS Measure the number of I/O s per second. Consider adding more data disks in line with your IOPS requirements. Average disk sec / read Average disk sec / write Latency Measure disk latency. Note: Numbers might vary; look at averages over time. Average disk bytes / read Average disk bytes / write Block size Measure the size of I/O s being issued. Note: Larger I/O s tend to have higher latency, such as those associated with BACKUP/RESTORE. Average / current disk queue length Outstanding or waiting IOPS Provides insight into the applications I/O pattern. Disk read bytes/sec Disk write bytes/sec Throughput or aggregate throughput Measure of total disk throughput. Note: Ideally, larger block scans should be able to heavily utilize connection bandwidth (for example, your throughput can be higher with a smaller number of larger IOPS).

10 Windows Azure Storage Analytics Metrics Tracks aggregated storage usage for Blobs, Tables and Queues Capacity e.g. #containers, total #blobs Requests - #requests, total ingress/egress, average E2E latency and server latency, total # failures by category, etc. Access via storage account namespace Enable in portal or using Set Blob Service Properties (REST API) Set retention policy See Windows Azure Storage Metrics: Using Metrics to Track Storage Usage

Storage Analytics format Key columns to correlated with VM perf counters: TotalRequests, TotalIngress, TotalEgress for transfers and bandwidth AverageE2ELatency

11 Storage Analytics format Key columns to correlated with VM perf counters: TotalRequests, TotalIngress, TotalEgress for transfers and bandwidth AverageE2ELatency and AverageServerLatency for blob/disk latency and peaks ThrottlingErrors to understand if we re issuing too many requests. Check how many blobs x the storage account.

12 Performance Charts on the WA Portal VM Dashboard Monitor tab for storage account Enabled under the configure tab VM read and write to their VHDs using GetBlob and PutPage commands respectively

13 Storage options 13

14 Azure Storage All disks presented to Microsoft Azure VMs are VHD stored as page blobs Shared service, not a conventional disk subsystem No direct perf correlation between a disk request made by a VM and that of a physical hard disk Storage account scalability targets; No performance SLA Total Capacity Total Request Rate Total Bandwidth (Georedundant) 500 TB 20,000 / sec 10 gb/sec Ingress 20 gb/sec Egress Total Bandwidth (Locally Redundant) 20 gb/sec Ingress 30 gb/sec Egress

Azure Data Disks Disks implemented as a shared multitenant service Built-in triple redundancy, optional geo-redundancy Performance less predictable than onprem Host machines, storage services,

15 Azure Data Disks Disks implemented as a shared multitenant service Built-in triple redundancy, optional geo-redundancy Performance less predictable than onprem Host machines, storage services, network bandwidth shared between subscribers Perf can depend on where and when VM is provisioned Subject to maintenance operations Granular control & configurability vs. cost, simplicity, out of box redundancy LB Front-ends Partition Layer Stream Layer Intra-stamp replication Storage Stamp Storage Location Service Geo-replication LB Front-ends Partition Layer Stream Layer Intra-stamp replication Storage Stamp

16 Azure Data Disk performance Different behavior compared to physical spindles Not a significant difference between random/sequential Cache should be turned off for disks hosting database files Sample SQL IO Measurement tests for single disk: Random I/O (8KB Pages) Sequential I/O (64KB Extents) Sequential I/O (256KB Blocks) Reads Writes Reads Writes Reads Writes IOPS Bandwidth 4 MB/s 4 MB/s 30 MB/s 20 MB/s 70 MB/s 70 MB/s

17 Multiple Disk Configuration Disk Striping with Windows Server 2012 (R2) Storage Spaces Number of Columns == number of disks in the pool Interleave == 64KB (256KB for DW optimized pools) Two main configuration options Option 1: Single virtual disk built on a pool with all available Data Disks Place all data and log files for user and system databases, simplified configuration Option 2: Create multiple Storage Pools and volumes (4-8 Data Disks each) and separate data and log files based on IO workload needs Example: 8-disks pool for user data files (up to 4000 IOPs / 200 MB/sec) 4-disks pool for user log files (up to 2000 IOPs / 100 MB/sec) 4-disks pool for TempDB (data + log)

18 Aggregated Measurement Samples (A7 VM) 4 disks Random I/O (8KB Pages) Sequential I/O (64KB Extents) Sequential I/O (256KB Blocks) Reads Writes Reads Writes Reads Writes IOPS Bandwidth 20 MB/s 20 MB/s 120 MB/s 80 MB/s 170 MB/s 270 MB/s 8 disks Random I/O (8KB Pages) Sequential I/O (64KB Extents) Sequential I/O (256KB Blocks) Reads Writes Reads Writes Reads Writes IOPS Bandwidth 30 MB/s 30 MB/s 150 MB/s 160 MB/s 170 MB/s 550 MB/s 16 disks Random I/O (8KB Pages) Sequential I/O (64KB Extents) Sequential I/O (256KB Blocks) Reads Writes Reads Writes Reads Writes IOPS Bandwidth 60 MB/s 60 MB/s 150 MB/s 300 MB/s 170 MB/s 600 MB/s

19 Aggregated Measurement Samples A7 16 disks Random I/O (8KB Pages) Sequential I/O (64KB Extents) Sequential I/O (256KB Blocks) Reads Writes Reads Writes Reads Writes IOPS Bandwidth 60 MB/s 60 MB/s 150 MB/s 300 MB/s 170 MB/s 600 MB/s A9 16 disks Random I/O (8KB Pages) Sequential I/O (64KB Extents) Sequential I/O (256KB Blocks) Reads Writes Reads Writes Reads Writes IOPS Bandwidth 60 MB/s 60 MB/s 340 MB/s 320 MB/s 320 MB/s 320 MB/s

20 Data and log files as blobs in Azure Storage (XI)

21 Recommendations Move user and system database files to Data Disks Turn off disk cache Create storage pools for data, logs and tempdb files Use general purpose guidance for contention Leverage data compression where possible Instant file initialization is more important than on premises 21

22 Backup and Restore 22

database administrators new to Azure and for ad hoc scenarios Complexity hidden Detailed Azure knowledge not

23 Deploy databases to Microsoft Azure VM What s being delivered New wizard to deploy databases to SQL Server in Windows Azure VM Can also create a new Windows Azure VM if needed Main benefits Easy to use Perfect for database administrators new to Azure and for ad hoc scenarios Complexity hidden Detailed Azure knowledge not needed Almost no overhead: defining factor for time-to-transfer is database size CloudAdapter helps database deployment

Backup to URL (SQL2012/SQL2014) Windows Azure VM Microsoft Azure Storage Near bottomless storage Off-site, geo-redundant No provisioning No device management Media safety (decay free)

$core.windows.net/backups/db.bak" $server = "SQLSERVER:\SQL\[computer]\DEFAULT" CD $server BACKUP DATABASE mydb TO URL ='https://mystorage.blob.core.windows.net/backup-container/mydbmybackup.$ credentialname = strcredential; 20130411.bak' mybackup.database Backup-SqlDatabase = strdatabase; -Database AdventureWorks2012 WITH mybackup.

credentialname = strcredential; 20130411.bak' mybackup.database Backup-SqlDatabase = strdatabase; -Database AdventureWorks2012 WITH mybackup.

24 Backup to URL (SQL2012/SQL2014) Windows Azure VM Microsoft Azure Storage Near bottomless storage Off-site, geo-redundant No provisioning No device management Media safety (decay free) Remote accessibility On-premise box CREATE using Microsoft.SqlServer.Management.Smo; CREDENTIAL mystoragecred WITH IDENTITY = mystorage', SECRET $credential = "mycredential" = <your storage access key> Backup mybackup = new Backup(); $url = " $server = "SQLSERVER:\SQL\[computer]\DEFAULT" CD $server BACKUP DATABASE mydb TO URL =' = strcredential; bak' mybackup.database Backup-SqlDatabase = strdatabase; -Database AdventureWorks2012 WITH mybackup.compressionoption CREDENTIAL = mystoragecred', -backupfile $url = BackupCompressionOptions.On; FORMAT, mybackup.devices.adddevice(desturl, COMPRESSION, STATS = 5, DeviceType.Url); MEDIANAME mybackup.sqlbackup(mylocalserver); -SqlCredential = mydb $credential ', -CompressionOption MEDIADESCRIPTION = On'Backup of mydb'

25 Backup to Microsoft Azure Simple configuration UI Easy creation of Azure credential No overhead

26 Backup to Microsoft Azure Tool What is it? Stand-alone Tool that adds backup to Microsoft Azure capabilities and backup encryption to prior versions of SQL Server Benefits One Cloud Backup strategy across prior versions of SQL Server including 2005, 2008, and 2008 R2 Adds backup encryption to prior versions, locally or in the cloud Takes advantage of backup to Azure Easy configuration

27 Managed Backup to Microsoft Azure What does it deliver An agent that manages and automates SQL Server backup policy Main benefits Flexibility in configuration and no need to manage backup policy Minimal knobs control retention period Context-aware e.g. workload/throttling Manage whole instance, or particular DBs Leverage Backup to Azure Inherently off-site & geo-redundant Minimal storage costs Zero hardware management Intelligence built-in Mainloop Retention Example: @enable_backup=1 GO

28 High Availabiliy and Disaster Recovery 28

29 SQL Server HA/DR Scenarios Availability of SQL Server in Azure VM Protect from issues impacting SQL Server or Azure VM Use a replica SQL Server in another Azure VM in same Region Availability of SQL Server in Azure VM Protect from issues impacting the Azure data center Use another SQL Server VM in different Azure DC Availability of on-premise SQL Server (physical or virtual) Protect from issues impacting on-premise data center Use a replica SQL Server in a Microsoft Azure VM

30 SQL Server HA within an Azure Region SQL Server service could be down or hung Patching guest OS causes downtime Patching SQL Server causes downtime ~15 min downtime ~12 min downtime

31 What to do? Configure a SQL Server HA technology within an Azure region Witness Failure detection for SQL Server Failover takes ~10s VM PS PS VM VM

32 What to do? SQL Server HA Technologies Technology SQL Server Version Zero Data Loss Scope # Replicas Automatic Failover Availability Groups SQL Server 2012 SQL Server 2014 Yes (Sync Mode) DB(s) 4 9 Yes (w/ additional cluster member) Readable Secondaries Yes Database Mirroring SQL Server 2008 R2 SQL Server 2012 SQL Server 2014 Yes (High Safety Mode) DB 1 Yes (w/ Witness) Limited (database snapshots)

33 Before starting HA configuration Ensure VMs are hosted appropriately Different Fault Domain and Upgrade Domain Ensures that a rack failure or Azure s upgrade impacts at most one replica Maintain VM IPs after Azure s service healing Avoids DNS update time: ~12 min Same Azure cluster Reduces log synchronization latency

34 Before starting HA configuration Prepare the VMs Setup an Active Directory Domain Controller Join VMs to the Domain Gotcha Azure s DHCP assigns a dup IP to the cluster network name (CNN) Can cause cluster communication issues Workaround Temporarily assign a link-local IP (e.g ) to CNN during creation Use the CreateAzureFailoverCluster script Download

35 Setup HA configuration Same as on-premise Tutorial:

36 AlwaysOn Listener configuration Step 1- configure endpoint on Azure VMs inside the Cloud Service Hotfix required for win 2008R2/2012

37 AlwaysOn Listener configuration Step 2- create listener using Failover Cluster manager and add the Dependency Then, manually modify it using powershell

38 AlwaysOn Listener configuration Step 3- set TCP listener port using SSMS Only one Listener per Cloud Service allowed Troubleshooting guide :

39 SQL Server DR between Azure Regions VM recovery: ~24h Data loss: ~30 minutes Azure s Geo-Replication doesn t guarantee write order across disks Can break SQL Server s recovery Failover to DR site and verify app

40 What to do? Configure a SQL Server DR technology between Azure regions

41 What to do? SQL Server DR Technologies Technology SQL Server Version Zero Data Loss Scope # Replicas Automatic Failover Availability Groups SQL Server 2012 SQL Server 2014 Yes* (Sync Mode) DB(s) 4 9 Yes** (w/ additional cluster member) Readable Secondaries Yes Log Shipping SQL Server 2008 R2 SQL Server 2012 SQL Server 2014 No DB 1 No Limited (standby state) * Most customers use Async Mode (High Performance) to avoid impacting primary performance **Async Mode only supports Force Failover to make DR (potential data loss) a conscious decision

Before starting DR configuration Configure VNET to VNET Connectivity Tutorial: http://msdn.

42 Before starting DR configuration Configure VNET to VNET Connectivity Tutorial: Secure site-to-site tunnel between Azure VPN gateways

43 Deployment topology

44 SQL Server DR between on-premise and Microsoft Azure Temporarily (e.g. Gateway or ISP failure) Permanently (e.g. flooding) Site rent + maintenance, Hardware, Ops

45 What to do? Configure a SQL Server DR technology between on-premise and Azure

46 What to do? SQL Server DR Technologies Technology SQL Server Version Zero Data Loss Scope # Replicas Automatic Failover Availability Groups SQL Server 2012 SQL Server 2014 Yes* (Sync Mode) DB(s) 4 9 Yes** (w/ additional cluster member) Readable Secondaries Yes Log Shipping SQL Server 2008 R2 SQL Server 2012 SQL Server 2014 No DB 1 No Limited (standby state) * Most customers use Async Mode (High Performance) to avoid impacting primary performance **Async Mode only supports Force Failover to make DR (potential data loss) a conscious decision

47 Before starting DR configuration Configure site-to-site tunnel between on-premise and Azure Tutorial:

48 Deployment topology

Add AlwaysOn replica in Microsoft Azure What s being delivered Wizard to add a replica in a Windows Azure VM Main benefits It takes care of: VM Creation (including data disks and Network Config)

49 Add AlwaysOn replica in Microsoft Azure What s being delivered Wizard to add a replica in a Windows Azure VM Main benefits It takes care of: VM Creation (including data disks and Network Config) Joining VM to Domain and Cluster Adding Node to Availability Group Database backup/restore and AG configuration You need to manually configure the listener 06/sql-server-2014-alwayson-availability-groups-add-azurereplica-wizard-in-action.aspx - Tutorial On-Premises 49

50 Questions?

51 Thank You for Attending

SQL Server 2014 Private and Hybrid Cloud Features. Darmadi Komo Senior Technical Product Marketing Mgr, SQL Server Product Mgmt

SQL Server 2014 Private and Hybrid Cloud Features Darmadi Komo Senior Technical Product Marketing Mgr, SQL Server Product Mgmt Agenda Why Microsoft Data Platform? Better Together: SQL Server, Windows Azure,