SQL Server Databases in the Clouds

Size: px

Start display at page:

Download "SQL Server Databases in the Clouds"

Shanon Banks
5 years ago
Views:

1 SQL Server Databases in the Clouds SQL SATURDAY # 787 TIM PLAS AUG 18, 2018 Virteva, Inc. 2015

& ServiceNow Current specialties: Cloud architecture / engineering, operational DBA services, HA/DR, server

2 Virteva, Inc The presenter Tim Plas Principal Consultant at Virteva (Mpls) IT infrastructure consulting & services, & ServiceNow Current specialties: Cloud architecture / engineering, operational DBA services, HA/DR, server migrations 30+ years in IT, in many roles Mostly servers & infrastructure in last 25 years CTO / co-founder of successful hosting company Multiple MS certs starting in tplas@cloudnet.com

3 The agenda: Cloud-based Microsoft SQL Server --A survey of the options Focused on MS SQL Server here Which clouds? & what options are available in each? Approaches being examined: IaaS Traditional SQL Server on a VM PaaS Platform as a Service (DB as a Service) Considerations / trade-offs Choosing options SQL Server in the Clouds 3

4 What does Cloud mean? Your data lives outside your premises Someone else owns the servers Mostly it s about the business model: You re renting server capacity by the slice NIST cloud definition: On-demand, wide access, resource pooling, elastic, measured ** ** SQL Server in the Clouds 4

5 Two main options for cloud SQL IaaS Infrastructure as a Service The way we ve always done it, with VM s PaaS Platform as a Service (or DBaaS) Google GCP Only SQL Server option is IaaS Amazon AWS RDS for SQL Server (Relational DB Service) Microsoft Azure SQL Database Multiple flavors & variants SQL Server in the Clouds 5

6 IaaS / PaaS differentiators Are you renting servers or services? Who maintains the O/S? Patching? Updates? Who handles HA/DR? And how? How much access do I have / do I need? Server/instance level? DB-level only? Performance options Available functionality: DB size, Service Broker, SSRS, SSIS, SSAS Costs! -both rental & maint. SQL Server in the Clouds 6

7 IaaS: InfraStructure as a Service Running a Windows VM with SQL Server installed, in a cloud provider s data center Not much different than running SQL VM in your own data center, or a co-lo data center Should do some config things differently Can spin up server image with pre-installed SQL, or do your own installation of SQL onto a VM Can BYOL (Bring Your Own License), or rent SQL license SQL Server in the Clouds 7

8 AWS RDS for SQL Server RDS = Relational Database Service AWS has several flavors of RDS -- Oracle, MySQL, MariaDB, Amazon Aurora, PostgreSQL, Microsoft SQL Rent as a service: Just a SQL instance, no server/os SQL Server 2008-R2, 2012, 2014, 2016, 2017 Enterprise or Standard Edition (or Web or Express) Out-of-the-box SQL Server, with special security lock-downs Can do geo-separated HA (using DB mirroring?) SQL Server in the Clouds 8

9 Azure options for SQL Server services IaaS: VM with boxed version of SQL Server Azure SQL Database Single DB, or Elastic Pool Azure SQL Managed Instance Azure SQL Data Warehouse Azure Analysis Services (like SSAS) Azure Data Factory for SSIS No SSRS-as-a-service SQL Server in the Clouds 9

10 Azure SQL Database service Rent as a service: Just a database, no server Comes with Enterprise Edition features, & built-in High Availability Not quite same thing as out-of-the-box SQL Cloud first -- New SQL features now delivered first on cloud Can do geo-replication (to another region) Two options: Single Database or Elastic DB Pool Allowing dedicated or shared resources amongst DB s SQL Server in the Clouds 10

11 Azure SQL Managed Instance Coming soon Currently in Public Preview Similar concept to AWS RDS for SQL Azure options now are: VM level (IaaS) DB level (Paas: Azure SQL DB) SQL instance level (PaaS: Managed Instance) Some slight incompatibilities & differences in operation But generally easiest / quickest way to migrate SQL to Azure Service: Microsoft handles all backups, HA, DR, patching SQL Server in the Clouds 11

12 Azure SQL Data Warehouse Related to on-prem Parallel Data Warehouse (PDW) Massively Parallel Processing (MPP) Like Hadoop big-data processing, for SQL Uses PolyBase to query big-data sources; Can efficiently process petabytes of data Not precisely same thing as SQL Server; has differences in T-SQL, data types, etc. Great cost model: Turn on and off as needed SQL Server in the Clouds 12

13 Azure services for SSAS, SSIS, SSRS Azure Analysis Services SSAS-as-a-service Tabular model only --Not multi-dimensional (cube) Also consider: Azure Data Lake Analytics (ADLA), Azure Databricks Azure Data Factory Can perform SSIS type work, + can run actual SSIS packages SSRS Not available as a service Was offered at one time in preview, but withdrawn Incorporating into Power BI Service? SQL Server in the Clouds 13

14 Pricing & Service Tiers IaaS Simply VM sizing, & choice of installed software AWS RDS Basically same as VM sizing (plus SQL edition) Azure Multiple options: Capability (& performance) tiers: Basic / Standard / Premium Classes within tier (e.g., S1, S2, S3, P1, P2) Pricing model: vcore, DTU SQL Server in the Clouds 14

15 Comparing the options SQL versions / editions Application compatibility Backup / restore Snapshots Data import / export SQL replication HA / DR Connections Security Management Performance & scalability Other SQL components Licensing & Costs SQL Server in the Clouds 15

16 SQL versions & editions IaaS: Any version and edition you want Azure SQL DB: Special version of SQL Azure SQL Managed Instance: Very close to boxed SQL 2017 Azure SQL Data Warehouse: Close to SQL, but not full compat. AWS RDS: Out of-the-box SQL: 2008-R Standard or Enterprise (or Web or Express) Version upgrades in PaaS: User controlled SQL Server in the Clouds 16

17 Application compatibility No problem with IaaS: Just install SQL version N Azure SQL Database: V12 (V11 retired) Compatibility Level: default 140 AWS RDS: SQL 2008-R Typically not a problem for in-house apps Vendor apps might only support or certify (be tested on) certain SQL versions Compatibility / vendor support might hold up adoption of DBaaS SQL Server in the Clouds 17

18 Backups No such thing as backups; only recovery plans Disk protection is well covered in cloud Azure has never lost a disk Other reasons for SQL backups: HA / Continuous availability D/R (for site) Point-in-time restores, Object-level restores Archival Providing copies of data to others Dev / Test / QA copies Migrating data SQL Server in the Clouds 18

19 Traditional SQL backup/restore IaaS Can do normal SQL backup/restore Working with virtual disks Options to use cloud storage PaaS It s different No access to server file system Traditional SQL backup/restore not available Cloud provider has special backup mechanisms 3 rd party backup tools DR consider geo-replication SQL Server in the Clouds 19

20 Point-in-time & Object-level restores Both AWS RDS and Azure SQL DB have good point & click tools Tools allow easy P.I.T. selection for restore With Azure SQL DB, how far you can go back depends on service tier For object-level restore, restoring to new (temp) DB alongside original, then pick off the piece(s) you want SQL Server in the Clouds 20

21 Snapshots, e.g. reporting AWS RDS has database snapshots Both on-demand and automatic AWS uses disk-level snapshots heavily Azure SQL DB has Database Copy Like snapshot (almost instantaneous), but has billable implications Azure SQL Managed Instance No snapshot; Copy-only backups Full / T-log backups are system managed SQL Server in the Clouds 21

22 Getting copies of whole DB For Dev / Test / QA / Reporting For AWS, strongly consider snapshots, for IaaS and PaaS (dev/test) With PaaS, need other methods for archiving / transmittal copies Azure PaaS: Azure SQL Database: COPY-DB Managed Instance: Copy-Only backup Azure SQL Data Warehouse: Only system managed; maybe ADF? SQL Server in the Clouds 22

23 Data migration / import / export No native-sql backup/restore with most PaaS No access to disk drives Lots of other methods available: DAC BACPAC; Generate a script BCP; SSIS Deploy to Azure SSMS Copy-Database Azure SQL Data Sync SQL Server Migration Assistant (SSMA) SQL Server in the Clouds 23

24 SQL replication Azure SQL Database Can be subscriber (target) for replication Consider use of Azure Data Sync Azure SQL DB as hub Azure Active Geo-Replication: Read-only secondaries Azure SQL Managed Instance: NO replication (but still Preview) AWS RDS: Can be target of replication, for migration purpose SQL Server in the Clouds 24

25 High Availability / Continuous availability IaaS: Have to build HA yourself Not trivial Failover clustering (FCI), SQL Availability Groups (AG) Used to use DB Mirroring (but deprecated) No shared-disk in cloud for clustering Can do FCI with replicated disk, e.g. SIOS, Windows S2D Can use File Share Witness or Cloud Witness for AG cluster No promiscuous ARP in cloud to switch IP addresses Have to work around with cloud load-balancers PaaS: HA built in; part of the service SQL Server in the Clouds 25

26 Disaster Recovery Cloud providers do have outages Arguably less frequent than most on-prem data centers, but still happens Ultimately still using physical devices in a physical location Cloud providers in-site availability is very good whole site goes down? Need to really think about D/R across multiple sites but what if their $$$ to run a fully redundant 2 nd site (incl. data transfer charges) Easy to protect data; harder to protect app Azure Paired Regions leverage it for DR SQL Server in the Clouds 26

27 Connection considerations Connect with same mechanisms as on-prem: SSMS, SQLCMD, Linked Server, ODBC, ADO.NET conn. strings, SSIS, SSRS, etc. Authentication: PaaS doesn t allow simple Windows Auth Azure PaaS can now use Azure AD AWS RDS can use Windows Auth, w AWS Managed AD & trusts Latency: Between cloud data centers, or between cloud and local premises Session timeouts different; need re-try logic SQL Server in the Clouds 27

28 Security considerations Cloud services have own firewalls and firewall rules Azure Service Endpoints feature for non-public PaaS Private networks (typically VPN for IaaS) Express Route with Azure, Direct Connect with AWS Encryption with PaaS: Data in flight: SSL encryption Data at rest: TDE (needs Enterprise Edition on AWS) Data in use (Azure): Always Encrypted, Auditing, Data Masking, SQL Server in the Clouds 28

29 Managing SQL Cloud usage encourages scripted operations ( DevOps ) SSMS (SQL Management Studio), SQL Ops Studio, PowerShell Cloud portal: Azure or AWS Monitoring & alerting Azure: Azure Diagnostics, service monitoring System Center, OMS, Azure Monitor AWS: Cloud Watch SCOM Mgmt Pack for AWS also available Many DMV s are available for PaaS, but not all SQL Server in the Clouds 29

30 Performance considerations CPU / Compute power, memory, network, DISK IOPS! Provisioned IOPS in AWS Adding disk volumes & striping SSD storage, &/or Ephemeral disk Data disks TempDB SQL 2014 BPE Simpler to manage with Azure PaaS DTU model: Transactions/sec SQL Server in the Clouds 30

31 Performance considerations AWS RDS SQL Server: Instance class Need to size like you would a SQL VM (complex) Azure SQL Database: Service Tier Basic / Standard / Premium DTU Database Throughput Units; edtu, DWU Transactions per hour / minute / second DTU is roughly equivalent to trans/sec Actual example of utility computing or vcore pricing SQL Server in the Clouds 31

32 Scalability considerations Cloud: Just turn the dial IaaS: Same as physical (or virtual) hosts: Scale up AWS RDS approach: Scale up Azure SQL DB advice used to be Scale-out (Sharding) Now also Scale-up, thru various performance tiers Max DB size for Azure SQL DB: 4 TB (varies by tier / perf. level) Max DB s / instance: 30 for AWS RDS Scaling storage: Now supported for both AWS & Azure SQL Server in the Clouds 32

33 Components besides DB engine System DB s: No (or very little) access w PaaS SQL Agent with PaaS: Azure: Elastic DB Jobs on Azure SQL DB Azure SQL Managed Instance: SQL Agent, w limitations Supported on AWS RDS SQL, w special considerations Can run regular SQL instance for SSIS / SSRS / SSAS, or SQL Agent, connected to PaaS DB s --but use services if possible SQL Server in the Clouds 33

34 Licensing considerations Standard disclaimer: IANAL (I Am Not A Lawyer) Renting licenses (hosting model), vs. allocating your owned licenses to another location BYOL = Bring Your Own License --vs. License Included --Software Assurance (SA) is a big deal DR licensing rules may still work for you Active/Passive set AWS has option for Dedicated hosts Azure now offering constrained-cores for core-based licensing SQL Server in the Clouds 34

35 Cost considerations IaaS pricing model: Per Windows VM AWS RDS pricing model: Per SQL instance ( = VM) Priced like VM model; Max 30 DB s / instance Azure SQL Database pricing model: Per DB (or pool), or vcore Performance tiers: Basic, Standard, Premium DTU s ; 2GB - 4TB Elastic DB (for pool of DB s) --edtu more expensive, but shareable Azure Managed Instance like AWS RDS: Per-Instance Ingress/Egress fees: Free inbound, $$ outbound SQL Server in the Clouds 35

36 Which option makes sense? It all depends --Which factors are important to you? Probably already decided on cloud provider So IaaS or PaaS? Will PaaS work for you? Building new app, or moving existing? Need special SQL features? (SSRS, SSAS, SB ) How do you plan to handle HA and DR? SQL Server in the Clouds 36

37 My recommendations Strongly consider PaaS if possible esp. if new app - ( Buy vs. build decision) Good HA & DR is much harder to build than buy PaaS is likely to be more cost-effective, both in hard cost (per-month service), & people costs Best advice- Whatever you pick, be sure to measure it and adjust it to suit actual needs - Because CLOUD! SQL Server in the Clouds 37

38 Questions? Tim Please turn in eval s; Visit the sponsors SQL Server in the Clouds 38

Successfully migrate existing databases to Azure SQL Database. John Sterrett Principal Consultant

Successfully migrate existing databases to Azure SQL Database John Sterrett Principal Consultant Common Migration failures Plan Test Execute Manage Free Reference Material Videos, Reference Links, Tips,