Aurora, RDS, or On-Prem, Which is right for you

Size: px

Start display at page:

Download "Aurora, RDS, or On-Prem, Which is right for you"

Gavin Mathews
5 years ago
Views:

1 Aurora, RDS, or On-Prem, Which is right for you Kathy Gibbs Database Specialist TAM Santa Clara, California April 23th 25th, 2018

2 Agenda RDS Aurora EC2 On-Premise Wrap-up/Recommendation Resources

SQL Server Automated provisioning, patching, scaling, replicas,

3 Amazon RDS Managed Relational Database Service in the AWS cloud Multi-engine support: Amazon Aurora, MySQL, MariaDB, PostgreSQL, Oracle, SQL Server Automated provisioning, patching, scaling, replicas, backup/restore Easily scales to handle growth High availability with Multi-AZ and Amazon Aurora

RDS MySQL Most popular open-source database engine Support for MySQL Community Edition versions 5.5, 5.6 and 5.7 InnoDB and MyISAM storage engines Version 5.

4 RDS MySQL Most popular open-source database engine Support for MySQL Community Edition versions 5.5, 5.6 and 5.7 InnoDB and MyISAM storage engines Version New Features JSON support Query optimizer improvements GIS extensions Improved parallel replication Dynamic buffer pool resizing Version 8.0 coming soon

from RDS MySQL XtraDB and Aria storage engines only Thread Pooling GTID Version 10.

5 RDS MariaDB Support for versions 10.0, 10.1, and 10.2 Same instance, regions, pricing as RDS MySQL (including free tier) Differences from RDS MySQL XtraDB and Aria storage engines only Thread Pooling GTID Version New Features InnoDB now default storage engine Multiple triggers on the same event Auto-partition of table cache

RDS Storage The type of storage can make a big difference for I/O intensive operations Magnetic Low cost but IOPS and latency can vary - Magnetic storage has average IOPS and latencies that are an

6 RDS Storage The type of storage can make a big difference for I/O intensive operations Magnetic Low cost but IOPS and latency can vary - Magnetic storage has average IOPS and latencies that are an order of magnitude slower than SSD-based storage. General Purpose (GP2) This is the default. SSD based storage with 3K IOPS burst capability then a baseline rate of 3 IOPS per GB. Throttled via a credit-based system. Great for storage below 1 TB especially when you do not deplete credits. Provisioned IOPS (PIOPS) SSD based storage with defined IOPS rates. Great when you need consistent performance or when you need very high performance. For almost all use cases we recommend an SSD-based storage type

RDS Scaling Scale compute/memory vertically up or down Handle higher load to grow over time Lower usage to control costs New host is attached to existing storage with minimal

7 RDS Scaling Scale compute/memory vertically up or down Handle higher load to grow over time Lower usage to control costs New host is attached to existing storage with minimal downtime Scale up Amazon EBS storage (now up to 16 TB!) Amazon EBS engines now support Elastic Volumes for fast scaling No downtime for storage scaling Can re-provision IOPS on the fly

RDS Replication Multi AZ Synchronous replication highly durable Only primary instance is active at any point in time Backups can be taken from secondary

Replica Asynchronous replication highly scalable All replicas are active and can be used for read scaling No backups configured by default Can be within an

8 RDS Replication Multi AZ Synchronous replication highly durable Only primary instance is active at any point in time Backups can be taken from secondary Always in two Availability Zones within a Region Database engine version upgrades happen on primary Automatic failover when a problem is detected Read Replica Asynchronous replication highly scalable All replicas are active and can be used for read scaling No backups configured by default Can be within an Availability Zone, cross- AZ, or cross-region Database engine version upgrades independently from source instance Can be manually promoted to a standalone database

RDS Backups Automated Backups Specify backup retention window per instance (7-day default) Kept until outside of window (35-day maximum) or instance is deleted Supports PITR Good for disaster

9 RDS Backups Automated Backups Specify backup retention window per instance (7-day default) Kept until outside of window (35-day maximum) or instance is deleted Supports PITR Good for disaster recovery Manual Backups Manually created through AWS console, AWS CLI, or Amazon RDS API Kept until you delete them Restores to saved snapshot Use for checkpoint before making large changes, non-production/test environments, final copy before deleting a database

RDS Security Amazon RDS is designed to be secure by default Network

Identity and Access Management (IAM)-based resource-level

or Oracle/Microsoft TDE No performance penalty for encrypting data

10 RDS Security Amazon RDS is designed to be secure by default Network isolation with Amazon Virtual Private Cloud (Amazon VPC) AWS Identity and Access Management (IAM)-based resource-level permission controls Encryption at rest using AWS KMS (all engines) or Oracle/Microsoft TDE No performance penalty for encrypting data Centralized access and audit of key activity Use SSL protection for data in transit

7 compatible with up to 5x better performance on the same hardware: 100,000 writes/sec and 500,000 reads/sec Scalable up to 64 TB

11 Amazon Aurora Amazon Aurora is a MySQL and PostgreSQL compatible relational database built for the cloud. Built from the ground up to leverage AWS MySQL 5.6 and MySQL 5.7 compatible with up to 5x better performance on the same hardware: 100,000 writes/sec and 500,000 reads/sec Scalable up to 64 TB in single database, up to 15 read replicas Highly available, durable, and fault-tolerant custom SSD storage layer: 6-way replicated across 3 Availability Zones Stored procedures in Amazon Aurora can invoke AWS Lambda functions

Aurora Management Amazon RDS manages your Amazon Aurora databases, handling time-consuming tasks such as provisioning, patching, backup, recovery, failure detection and repair.

12 Aurora Management Amazon RDS manages your Amazon Aurora databases, handling time-consuming tasks such as provisioning, patching, backup, recovery, failure detection and repair. Aurora s thread pooling works differently from MySQL s. Aurora s thread pool is much more scalable than MySQL s thread pool, handling over 5000 connection. As always, we recommend that you test your workload to make sure that you are getting the required performance

Aurora Scaling AWS Auto Scaling monitors your applications and automatically adjusts capacity to maintain steady, predictable performance AWS Auto Scaling

13 Aurora Scaling AWS Auto Scaling monitors your applications and automatically adjusts capacity to maintain steady, predictable performance AWS Auto Scaling makes scaling simple with recommendations that allow you to optimize performance, costs, or balance between them. Auto Scaling plans Aurora has auto-scaling storage

Aurora Storage Amazon Aurora delivers significant increases over MySQL performance by tightly integrating the database engine with an SSDbased virtualized storage

14 Aurora Storage Amazon Aurora delivers significant increases over MySQL performance by tightly integrating the database engine with an SSDbased virtualized storage layer purpose-built for database workloads Reducing writes to the storage system, Minimizing lock contention Eliminating delays created by database process threads.

Aurora Replication Amazon Aurora Replicas share the same underlying volume as the primary instance. Updates made by the primary are visible to all Amazon Aurora Replicas.

Data from your primary instance is replayed on your replica as transactions. With Aurora MySQL you can setup a cross-region Aurora Replica from the RDS console.

15 Aurora Replication Amazon Aurora Replicas share the same underlying volume as the primary instance. Updates made by the primary are visible to all Amazon Aurora Replicas. Replicas on the cluster will typically lag behind the primary by 10s of milliseconds You can also create MySQL Read Replicas based on MySQL s binlog-based replication engine. Data from your primary instance is replayed on your replica as transactions. With Aurora MySQL you can setup a cross-region Aurora Replica from the RDS console. Assign a promotion priority tier to each instance on your database cluster to get more control over replica promotion during failovers Failovers are typically under 30 seconds At re:invent 2017, we announced the preview for Amazon Aurora Multi-Master

16 Aurora Backups Automated backups are always enabled on Amazon Aurora DB Instances. Backups do not impact database performance. Will automatically attempt to recover your database in a healthy AZ with no data loss. Restore from a DB Snapshot or perform a point-in-time restore operation to a new instance. Note that the latest restorable time for a point-in-time restore operation can be up to 5 minutes in the past. Take Snapshots of the database with no performance impact. Keep these snapshots around as long as you want. Note that restoring data from DB Snapshots requires creating a new DB Instance.

17 Aurora Security/Encryption Amazon Aurora can encrypt data in transit through the AWS Key Management Service. Automated backups, snapshots, data at rest in the underlying storage and Replicas within the same cluster are also encrypted. Aurora database instances are created within an Amazon VPC, allowing users to isolate a database within their network for more security

18 Amazon EC2 Elastic Compute Cloud. Provides scalable computing capacity in the Amazon Web Services (AWS) cloud. Using Amazon EC2 eliminates your need to invest in hardware up front, so you can develop and deploy applications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage. Amazon EC2 enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your need to forecast traffic.

19 On-Premise Straightforward. This is utilizing hardware in your data center

20 After this, what is the conclusion?

21 Why use Amazon RDS? Lower TCO because we manage the muck Get more leverage from your teams Focus on the things that differentiate you Push button provisioning, automated instance and storage scaling, patching, upgrades, security, and general care and feeding Need or Want to Use MariaDB Ability to use the free tier to test the database Built-in high availability and cross-region replication across multiple data centers Managed disaster recovery with to the minute point-in-time recovery Managed read scaling through read replicas Even a small startup can leverage multiple data centers to design highly available apps with over 99.95% availability

22 Why use Amazon Aurora? Build on RDS so you get many of the features provided by the service Need Managed Database that is larger than 16TB, goes to 64 TB Fast Replication and Easy Failover Recovery from physical storage failures is transparent and instance restarts typically require less than a minute Have a highly transaction database with concurrent processes. Aurora can be quite a bit faster. Aurora excels in high concurrency scenarios, so if your workload involves lots of concurrent queries, Aurora may be a big upgrade from a performance standpoint. Have either MySQL or PostgreSQL Stored procedures in Amazon Aurora can invoke AWS Lambda functions Database restart is less than 60 seconds in most cases.

23 Why use Amazon EC2? Have a MySQL version requirement. Have a MySQL engine requirement Need to have access to hardware below the database Scripting Need software installed (i.e. Agents) O/S Configuration Have any limitation mentioned, but still want (need) to operate in the cloud Utilize benefits of the cloud Want to install additional software MySQL Enterprise Backup, Percona Cluster Consider using EBS volumes vs. ephemeral disks

24 Why use On-Premise? A requirement of the business Security software that is on-premise can be difficult to federate to the cloud You need to install a lot of agents for different applications standard in your business

25 Getting started with Amazon RDS for MariaDB Information Pricing MariaDB user guide t/userguide/chap_mariadb.html

26 Getting started with Amazon RDS for MySQL Information Pricing MySQL user guide /UserGuide/CHAP_MySQL.html

27 Getting started with Amazon Aurora Information Pricing Aurora user guide /UserGuide/CHAP_Aurora.html

28 Rate My Session 28

29 Thank You! Kathy Gibbs, Database Specialist Technical Account Manager,

30 Managed MySQL Compatible Engines at AWS Amazon Aurora Standard Community Performance The open source standard MySQL The popular community choice The fastest MySQL compatible engine 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Configuring your MySQL instance for data

(disables binlog) Turn off autocommit mode

31 Configuring your MySQL instance for data loading Turn off backup retention (disables binlog) Turn off autocommit mode Drop indexes and disable foreign keys For EBS-based engines Use EBS-optimized instance types Maximize storage IOPS Optimize parameter settings innodb_flush_log_at_trx_commit, innodb_io_capacity, innodb_io_read_threads, innodb_io_write_threads, sync_binlog Remember to re-enable settings after load completes!

Amazon RDS software patches are usually performed without restarting databases Database engine upgrades require downtime Minor version

32 How do you maintain my d atabase? H o w c a n I p l a n f o r i t? Any maintenance that causes downtime (typically only a few times per year) will be scheduled in your maintenance window Operating system or Amazon RDS software patches are usually performed without restarting databases Database engine upgrades require downtime Minor version upgrades automatic or manually applied Major version upgrades manually applied Version deprecations three- to six-month notification before scheduled upgrades View upcoming maintenance events in your AWS Personal Health Dashboard

migrations/database consolidation) Import Backup from Amazon S3 New for RDS MySQL Create full or incremental backup with

33 Migrating your MySQL Database into AWS Data Import Options Use mysqldump and mysql command line Use mysqlimport on EC2 instance Use external replication into AWS for minimizing downtime Use AWS Database Migration Services (heterogeneous migrations/database consolidation) Import Backup from Amazon S3 New for RDS MySQL Create full or incremental backup with Percona XtraBackup 2.3 Use AWS IAM role to access S3 bucket Use replication to catch up to changes in the source database (if necessary)

BERLIN. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

BERLIN. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved BERLIN 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Amazon Aurora: Amazon s New Relational Database Engine Carlos Conde Technology Evangelist @caarlco 2015, Amazon Web Services,