Using SQL Server on Amazon Web Services

Size: px

Start display at page:

Download "Using SQL Server on Amazon Web Services"

Gladys May
6 years ago
Views:

1 Using SQL Server on Amazon Web Services High Availability and Reliability in the Cloud Michael Barras, Sr. Database Engineer August 26, , Amazon Web Services, Inc. or its Affiliates. All rights reserved.

2 The big question: Have you tried deploying 100 SQL Server database instances in a few minutes or doing a point in time restoration with a few clicks? Amazon Relational Database Service for Microsoft SQL Server (RDS SQL Server) Deployment, management and configuration at scale Deployment patterns, identity management Backup/restore & data movement

3 Key tenets for success SQL Server as Managed Service No Data Lock-in Automation, Configuration at Scale

4 Amazon Web Services (AWS)

5 What is AWS? Utility Computing A broad and deep platform helps customers build sophisticated, scalable applications

6 AWS lets you focus on your business 30% 70% Traditional Data Center Your Business Managing All of the Heavy Lifting AWS Cloud-based Services More Time to Focus on Your Business 70% Configuring Your Cloud Assets 30%

7 Increased agility has become the #1 reason businesses use the AWS cloud

Availability Zones (AZs) (+14) Multiple, isolated locations within

8 The AWS global infrastructure 16 Regions Worldwide (+5 coming soon) Geographic areas where AWS services are available 44 Availability Zones (AZs) (+14) Multiple, isolated locations within a region Choose which regions and availability zones to deploy your workloads in

9 Breadth and depth of AWS services

10 No Data Lock-in SQL Server as Managed Service Automation, Configuration at Scale Awesome, how can I run SQL Server workloads on AWS?

11 Choose the best path for your needs

12 Choose the best path for your needs ü Managed physical infrastructure ü Managed DB install and backups ü Managed OS and patching ü Managed high availability and scaling ü Managed physical infrastructure ü Managed OS installation ü Managed scaling ü OS level control

13 Your responsibility App optimization, tuning Deployment Monitoring App optimization, tuning Deployment Monitoring High availability Backups DB & OS patching

14 Which one is right for you? Amazon RDS for SQL Server Consider RDS first Focus on: Business value tasks High-level tuning tasks Schema optimization Little in-house database expertise SQL Server on Amazon EC2 Need full control over: DB instance Replication Failover Clustering AlwaysOn Availability Groups Use options not in Amazon RDS

15 SQL Server features at a glance Amazon RDS for SQL Server SQL Server on Amazon EC2 Versions Supported: 2008 R2, 2012, 2014, 2016 All** Editions Supported: High Availability: Express, Web, Standard, Enterprise AWS-managed All** Self-managed; AlwaysOn, Mirror, Log Ship Encryption: Authentication: Encrypted Storage using AWS KMS (all editions); TDE Support Windows & SQL authentication Backups: Maintenance: Managed automated backups Automatic software patching Maintenance plans & 3 rd party tools Self-managed * Self-installed

16 No Data Lock-in SQL Server as Managed Service Automation, Configuration at Scale Deploying and operating SQL Server on AWS at scale using RDS for SQL Server

17 Deploy and manage SQL Server Multiple ways to start and manage your SQL Server resources using AWS Amazon RDS Management Console AWS Command Line Interface (CLI) or AWS Tools for PowerShell AWS SDKs AWS CloudFormation templates

18 Automating deployments Launch RDS DB Instance Programmatically New-RDSDBInstance -DBInstanceIdentifier "demo-sqlsrv" -DBInstanceClass "db.r3.large" -Engine "sqlserver-se" -EngineVersion " v1" -AllocatedStorage 200 -StorageType "gp2 -LicenseModel "license-included" -MultiAZ 1 -DBParameterGroupName "demo-2014se" -AutoMinorVersionUpgrade 1 -MonitoringInterval 15 -MonitoringRoleArn "arn:aws:iam:..." -BackupRetentionPeriod 35 -Domain "d-xxxxxxxxxx"-domainiamrolename "rds-ds-access-role" -DBSubnetGroupName "demo-subnets" -VpcSecurityGroupId "sg-xxxxxxxx -MasterUsername "demoadmin"-masteruserpassword "demopassword" -StorageEncrypted 1

19 Automating deployments Launch RDS DB Instance Programmatically New-RDSDBInstance -DBInstanceIdentifier "demo-sqlsrv" -DBInstanceClass "db.r3.large" -Engine "sqlserver-se" -EngineVersion " v1" -AllocatedStorage 200 -StorageType "gp2 -LicenseModel "license-included" General & Performance -MultiAZ 1 -DBParameterGroupName "demo-2014se" -AutoMinorVersionUpgrade 1 -MonitoringInterval 15 -MonitoringRoleArn "arn:aws:iam:..." -BackupRetentionPeriod 35 -Domain "d-xxxxxxxxxx"-domainiamrolename "rds-ds-access-role" -DBSubnetGroupName "demo-subnets" -VpcSecurityGroupId "sg-xxxxxxxx -MasterUsername "demoadmin"-masteruserpassword "demopassword" -StorageEncrypted 1

20 Automating deployments Launch RDS DB Instance Programmatically New-RDSDBInstance -DBInstanceIdentifier "demo-sqlsrv" -DBInstanceClass "db.r3.large" -Engine "sqlserver-se" -EngineVersion " v1" -AllocatedStorage 200 -StorageType "gp2 -LicenseModel "license-included" -MultiAZ 1 -DBParameterGroupName "demo-2014se" -AutoMinorVersionUpgrade 1 -MonitoringInterval 15 -MonitoringRoleArn "arn:aws:iam:..." -BackupRetentionPeriod 35 Reliability & Tuning -Domain "d-xxxxxxxxxx"-domainiamrolename "rds-ds-access-role" -DBSubnetGroupName "demo-subnets" -VpcSecurityGroupId "sg-xxxxxxxx -MasterUsername "demoadmin"-masteruserpassword "demopassword" -StorageEncrypted 1

21 Automating deployments Launch RDS DB Instance Programmatically New-RDSDBInstance -DBInstanceIdentifier "demo-sqlsrv" -DBInstanceClass "db.r3.large" -Engine "sqlserver-se" -EngineVersion " v1" -AllocatedStorage 200 -StorageType "gp2 -LicenseModel "license-included" -MultiAZ 1 -DBParameterGroupName "demo-2014se" -AutoMinorVersionUpgrade 1 -MonitoringInterval 15 -MonitoringRoleArn "arn:aws:iam:..." -BackupRetentionPeriod 35 -Domain "d-xxxxxxxxxx"-domainiamrolename "rds-ds-access-role" -DBSubnetGroupName "demo-subnets" -VpcSecurityGroupId "sg-xxxxxxxx -MasterUsername "demoadmin"-masteruserpassword "demopassword" -StorageEncrypted 1 Domain Membership

22 Automating deployments Launch RDS DB Instance Programmatically New-RDSDBInstance -DBInstanceIdentifier "demo-sqlsrv" -DBInstanceClass "db.r3.large" -Engine "sqlserver-se" -EngineVersion " v1" -AllocatedStorage 200 -StorageType "gp2 -LicenseModel "license-included" -MultiAZ 1 -DBParameterGroupName "demo-2014se" -AutoMinorVersionUpgrade 1 -MonitoringInterval 15 -MonitoringRoleArn "arn:aws:iam:..." -BackupRetentionPeriod 35 -Domain "d-xxxxxxxxxx"-domainiamrolename "rds-ds-access-role" -DBSubnetGroupName "demo-subnets" -VpcSecurityGroupId "sg-xxxxxxxx -MasterUsername "demoadmin"-masteruserpassword "demopassword" -StorageEncrypted 1 Networking & Security

23 No Data Lock-in SQL Server as Managed Service Automation, Configuration at Scale Planning for performance with RDS for SQL Server

24 Service-level performance factors RDS DB Instance Class Compute Capabilities Memory Capabilities Network Performance Storage Performance vcpus GB of RAM MB/s (Throughput) I/O Throughput RDS Storage Type

25 Scaling compute resources R3 Range of DB instance classes From: 1 vcpu and 1 GB of RAM To: 40 vcpus or 244 GB of RAM Grouped in instance families: Standard (db.m4) Memory (db.r3) Burst Capable (db.t2) Scale up or down by changing the instance class

26 Performance planning SQL Server workloads typically benefit from large amounts of memory (caching) Consider db.r3 - Memory Optimized instances Edition and licensing may impact DB instance class options DB instances can be modified to change the DB instance class Requires a reboot (or failover in Multi-AZ) Can scale compute capacity with the workload, if practical Storage cannot be scaled once deployed Plan for data set growth

27 Storage & I/O performance Amazon RDS Amazon EC2 Type Size Performance Size Performance Burst Capacity Pricing Model Magnetic Storage General Purpose (SSD) Provisioned IOPS (SSD) 20 GiB 1 TiB ~100 IOPS 1 GiB 1 TiB ~ 100 IOPS Yes, several hundred IOPS 20 GiB 4 16 TiB (min. 100 GiB recommended) 100 GiB 4 16 TiB (min. 200 GiB for Standard edition & up) 3 IOPS/GiB 1 GiB 16 TiB 3 IOPS/GiB for volumes 1 TiB or less, up to 10,000 IOPS for larger volumes Up to max. 20,000 IOPS 4 GiB 16 TiB Up to 20,000 IOPS Yes, up to 3000 IOPS per volume, subject to credits (< 1 TiB in size) No, fixed allocation Allocated storage; I/O operations Allocated storage Allocated storage; Provisioned IOPS

28 SQL Server as Managed Service High availability and reliability No Data Lock-in Automation, Configuration at Scale

Amazon RDS Multi-AZ Always run production workloads in Multi-AZ mode Primary and secondary DB nodes in different Availability Zones (AZ) Leverages SQL Server DB Mirroring

29 Amazon RDS Multi-AZ Always run production workloads in Multi-AZ mode Primary and secondary DB nodes in different Availability Zones (AZ) Leverages SQL Server DB Mirroring Automatic failover (1 2 minutes typically) Requires 3 AZs (3rd AZ is for mirror witness) Consider: Impact on mirroring of change heavy workloads (for example, index rebuilds)

30 Amazon RDS Multi-AZ in depth Failure scenarios mitigated: Loss of availability in primary AZ Loss of network connectivity to principal DB node Compute unit or storage failure on principal DB node Failover process: Mirroring stopped Address apply debt Promote to master Change DNS endpoint Provision new secondary Consider: Implement retry logic at the application layer trigger manual failover to test

31 Monitoring SQL Server performance Monitor performance using Amazon CloudWatch Alarms & notifications: Amazon RDS & Amazon EC2 Default metrics: Amazon RDS & Amazon EC2 Custom metrics: Amazon EC2 CPU Utilization Read / Write IOPS Disk Queue Depth Memory (RDS) Storage Space (RDS) Connections (RDS) I/O Throughput (EC2) Use SQL Server Profiler & Tuning Advisor to trace query performance

32 Amazon RDS Enhanced Monitoring Overview: OS Level Monitoring Metrics 26 system and per process metrics Metrics delivered to CloudWatch Logs Up to 1 second granularity Compared to CloudWatch Metrics: Agent based metrics collections There can be differences with CloudWatch metrics due to collection source (hypervisor vs. agent) eg. CPU

33 Reliability Automated backup and recovery Maximum retention: 35 days Restore to any second, typically up to the last 5 minutes Full DB instance snapshots & restore Backup & Restore using.bak files Leverages SQL Server s native backup functionality Only full backups supported (no log shipping)

34 No Data Lock-in SQL Server as Managed Service Automation, Configuration at Scale Tuning and configuration management at scale

35 Manage the RDS SQL Server configuration Parameter Groups Centralized management of DB engine parameters Ability to consistently apply configurations to DB instances Auditability of configuration Sensible defaults work for most use cases Ability to create custom parameter groups Option Groups Used for enabling additional features Ability to create custom option groups Supported options: Transparent Data Encryption (TDE) in Enterprise Edition only S3 Backup & Restore

36 Customizing parameter groups

37 Customizing parameter groups Exercise Caution - Change at Your Own Risk! Not all parameters can be changed, some read only for visibility Dynamic (applied immediately) vs. Static (requires reboot) Fixed value, formula driven default, DB instance class dependent, interdependent for example: Parameter clr Enable (1) or disable (0) the common language runtime, default disabled (0). But if enabled, parameter lightweight pooling must be disabled (0) Parameter max server memory (mb) Memory allowed to be used by the server instance. Default based on instance class: {DBInstanceClassMemory}/ Parameter max worker threads Number of worker threads available for SQL Server processes. Default is 0 db engine computed based on formula: 512+ max(0, (vcpus-4)x16)

38 No Data Lock-in SQL Server as Managed Service Automation, Configuration at Scale Active Directory integration and deployment patterns

39 Using Windows Authentication Join RDS for SQL Server to a domain Domain provided by AWS Directory Services Directory as a managed service Deploy a Microsoft AD directory Fully managed AD forest Primary and secondary domain controllers in different AZs Ability to establish forest trusts

40 RDS for SQL Server deployment patterns Standalone DB Instance Microsoft AD integrated DB Instance Integration with existing Active Directory Infrastructure Hybrid onpremise and AWS deployment

Hybrid on-premise and AWS deployment Microsoft AD directory + on-prem AD Extend your internal network to AWS Private connectivity to your AWS VPC (VPN, DirectConnect)

41 Hybrid on-premise and AWS deployment Microsoft AD directory + on-prem AD Extend your internal network to AWS Private connectivity to your AWS VPC (VPN, DirectConnect) We recommend extending your AD deployment to AWS using secondary controllers in your VPC Establish Forest Trust between the existing AD and the Microsoft AD directory

42 No Data Lock-in SQL Server as Managed Service Automation, Configuration at Scale Networking and security with RDS SQL Server

43 Securing SQL Server on AWS: network 1 Amazon VPC: Control subnets, AZ specificity (DB subnet groups), route tables and NACLs 2 3 Security groups: Restrict instance traffic Public access: Avoid it or limit it VPC 4 Encrypt Traffic: Forced SSL supported

44 Securing SQL Server on AWS: data 1 Protect data at rest Encrypted DB instances using AWS KMS, TDE, column-level, encrypt before saving 2 Secure data in transit Encrypted connections via SSL, forced SSL supported

Securing SQL Server on AWS: access & audit 1 2 3 Grant least privileges to applications and end users Control: Use AWS Identity and

45 Securing SQL Server on AWS: access & audit Grant least privileges to applications and end users Control: Use AWS Identity and Access Management (IAM) to control instance lifecycle permissions, grant least privileges Audit: Use AWS CloudTrail to log AWS API invocations

46 No Data Lock-in SQL Server as Managed Service Automation, Configuration at Scale Enabling data access and movement

instance Amazon RDS does not provide desktop, Administrator or file-system access to DB

47 Amazon RDS SQL Server tooling Manage using common tools: SQL Server Management Studio, sqlcmd, etc. Data source or target only for SSAS, SSIS and SSRS Maximum 30 databases per Amazon RDS instance Amazon RDS does not provide desktop, Administrator or file-system access to DB instances Not supported: Maintenance Plans, Database Mail, MSDTC Limited support for Linked Servers available

Migrating data to & from Amazon RDS 1 2 3 4.

48 Migrating data to & from Amazon RDS BAK File Save & Restore Leverages SQL Server s native backup functionality Microsoft SQL Server Database Publishing Wizard, Import/Export Export to T-SQL files, load using sqlcmd AWS Database Migration Service Minimize downtime during migrations, migrate between different DB platforms, Schema Conversion Tool AWS Marketplace Third-party data import and export tools and solutions

49 Thank You

50 More Information Microsoft SQL Server on AWS Deploying SQL Server on AWS (whitepaper) Amazon RDS for SQL Server Supported Features Implement Linked Servers with Amazon RDS Implementing Microsoft Windows Server Failover Clustering and SQL Server AlwaysOn Availability Groups in the AWS Cloud

Aurora, RDS, or On-Prem, Which is right for you

Aurora, RDS, or On-Prem, Which is right for you Kathy Gibbs Database Specialist TAM Katgibbs@amazon.com Santa Clara, California April 23th 25th, 2018 Agenda RDS Aurora EC2 On-Premise Wrap-up/Recommendation