Introduction to Database Services

Size: px

Start display at page:

Download "Introduction to Database Services"

Katrina Small
5 years ago
Views:

2 Today s agenda Why managed database services? A non-relational managed database A relational managed database A managed in-memory cache A managed data warehouse

3 Why managed database services?

4 If you host your databases on-premises App optimization Scaling High availability Database backups DB s/w patches DB s/w installs OS patches OS installation Server maintenance Rack & stack Power, HVAC, net

5 If you host your databases on-premises App optimization Scaling High availability Database backups DB s/w patches DB s/w installs OS patches OS installation Server maintenance Rack & stack Power, HVAC, net

6 If you host your databases in Amazon EC2 App optimization Scaling High availability Database backups DB s/w patches DB s/w installs OS patches OS installation Server maintenance Rack & stack Power, HVAC, net

7 If you host your databases in Amazon EC2 App optimization Scaling High availability Database backups DB s/w patches DB s/w installs OS patches OS installation Server maintenance Rack & stack Power, HVAC, net

8 If you choose a managed DB service App optimization Scaling High availability Database backups DB s/w patches DB s/w installs OS patches OS installation Server maintenance Rack & stack Power, HVAC, net

9 The self-managed vs. AWS-managed decision Self-managed database (EC2) AWS-managed database Operations You have full responsibility for upgrades and backup AWS provides upgrades and backups as a service Security You have responsibility for security of the OS and above AWS provides security of the OS and DBMS installation and gives you tools to ensure DB security Control Full control over parameters of OS and database Database is a managed appliance, so you can easily automate High Availability Replication can be complex and requires a lot of operational support AWS provides failover as a packaged service

10 A Managed Service for Each Major DB Type Amazon DynamoDB Amazon RDS Amazon ElastiCache Amazon Redshift Document and key-value store SQL database engines In-memory cache Data warehouse

11 What is Amazon DynamoDB?

12 Amazon DynamoDB: a managed document and key-value store Simple and fast to deploy Simple and fast to scale (to millions of IOPS) Data is automatically replicated Fast, predictable performance (backed by SSD storage) Secondary indexes offer fast lookups No cost to get started; pay only for what you consume

13 Amazon DynamoDB is a schemaless database table items Attributes (name/ value pairs)

14 Each item must include a key CustomerID Top Scores Hash key (DynamoDB maintains an unordered index)

15 Each item must include a key CustomerID GameTitle Top Scores Hash key Range key (DynamoDB maintains a sorted index)

16 Local Secondary Indexes = alternate range keys GameTitle CustomerID TopScore Top Scores Hash key LSI Range key

17 Global Secondary Indexes = Pivot Charts for your table GSI GameTitle CustomerID TopScore

18 Amazon DynamoDB: provision throughput 100 Write capacity units 200 Read capacity units

19 DynamoDB: What are capacity units? 1 1 Pay to bearer on demand 1 write per sec of up to 1KB 1 1 One write capacity unit 1 1 Pay to bearer on demand 1 read per sec of up to 4KB 1 Eventually consistent reads at 50% off! 1 One read capacity unit

20 DynamoDB is optimized for developer productivity CreateTable PutItem Manage tables Query specific items OR scan full table UpdateTable DeleteTable GetItem UpdateItem DescribeTable ListTables DeleteItem Query BatchGetItem Scan BatchWriteItem Select, insert, update items Bulk select or update (max 1MB)

21 Simple app architecture with Amazon DynamoDB Business logic Elastic Load Balancing Amazon EC2 App Instances DynamoDB Clients

22 Simple app architecture with Amazon DynamoDB DynamoDB Clients

23 How DynamoDB billing works Monthly bill = GB + + Storage consumed (plus 100 bytes per item) 100 Write capacity units 200 Read capacity units 100 GB * $ * 720 hrs * $0.0065/ * 720 hrs * $0.0065/ $ Assumes DB instance accessed only from AWS region Further details at

24 What is Amazon RDS?

25 Amazon RDS: a managed SQL service Simple and fast to deploy Simple and fast to scale AWS handles patching, backups, replication Compatible with your applications Choose among Amazon Aurora, MySQL, PostgreSQL, Oracle, SQL Server Fast, predictable performance No cost to get started; pay only for what you consume

26 How Amazon RDS delivers high performance Choose General Purpose (SSD) storage for most workloads 3 IOPS per GB provisioned, with burst capability up to 3000 IOPS Choose Provisioned IOPS (SSD) storage for high, predictable performance Provision up to 3 TB storage and 30K IOPS per instance Scale IOPS up or down online Choose a database instance type with the right amount of CPU and memory

27 How Amazon RDS backups work Automated backups Restore your database to a point in time Enabled by default Choose a retention period, up to 35 days Manual snapshots Initiated by you Persist until you delete them Stored in Amazon Simple Storage Service (Amazon S3) Build a new database instance from a snapshot when needed

28 Choose Multi-AZ for greater availability An Availability Zone is a physically distinct, independent infrastructure With Multi-AZ operation, your database is synchronously replicated to another zone in the same AWS region Failover occurs automatically in response to the most important failure scenarios Planned maintenance is applied first to backup

29 Choose Read Replicas for greater scalability Offload read traffic to an automatically maintained read replica Create multiple read replicas, load-share traffic Easy to set up Available in Amazon RDS for MySQL and PostgreSQL RDS

30 Choose cross-region snapshot copy for even greater durability, ease of migration Copy a database snapshot to a different AWS region Warm standby for disaster recovery Or use it as a base for migration to a different region

31 Choose cross-region read replicas for enhanced data locality, even more ease of migration Even faster recovery in the event of disaster Bring data close to your customers Promote to a master for easy migration

32 How to scale with Amazon RDS Scale up or down with resizable instance types Scale your storage up with a few clicks while online Offload read traffic to read replicas Use a cache (more on this in a while) Amazon RDS takes some of the pain out of sharding

33 Amazon Aurora: High Availability by Default MySQL compatible Delivers 5x MySQL s throughput on the same hardware Your data is replicated 6 ways across 3 AZs Storage grows up to 64 TB seamlessly AZ 1 AZ 2 AZ 3 Aurora Virtualized, cross-az storage layer Aurora Aurora Aurora Aurora Up to 15 Aurora Replicas with instant crash recovery

34 How Amazon RDS billing works Monthly bill = GB + 4 vcpus; 15 GiB RAM db.m3.large; MySQL; Dublin; Single-AZ; On-Demand = 720 hrs * $ GB * $0.115 = $ GB General Purpose (SSD) Assumes DB instance accessed only from Amazon EC2 Further details at

35 What is Amazon ElastiCache?

36 ElastiCache: resizable in-memory cache High performance, resizable in-memory caching Speed your application by bypassing database access and disk storage Compatible with your existing applications Choose between the popular memcached and Redis engines

37 Use Cases for Amazon ElastiCache Performance or cost optimization of an underlying database Storage of ephemeral key-value data High-performance application patterns

38 Amazon ElastiCache: resilient app architecture AZ a Master Elastic Load Balancing AZ b Clients Read Replica

39 How Amazon ElastiCache billing works Monthly bill = N = 2 nodes * 720 hrs * $0.182 = $ m3.large; Oregon; On-Demand 2 vcpus; 7.1 GB RAM Further details at

40 What is Amazon Redshift?

41 Amazon Redshift: a managed data warehouse Petabyte-scale columnar database Fully managed, fault tolerant architecture Fast response time ~10x that of typical relational stores Pricing as low as $1,000 per TB per year

Companies Reduce costs by halting future HW expansion

functionality to applications Migrate completely from

Scale DW capacity as demand grows Respond faster to

42 Who uses Amazon Redshift? Traditional Enterprise DW Companies with Big Data SaaS Companies Reduce costs by halting future HW expansion Improve performance by an order of magnitude Add analytic functionality to applications Migrate completely from existing DW systems Make more data available for analysis Scale DW capacity as demand grows Respond faster to business; provision in minutes Access business data via standard reporting tools Reduce HW & SW costs by an order of magnitude

43 Amazon Redshift architecture Amazon S3 Existing business intelligence tools PostgreSQL JDBC/ODBC Leader node Amazon DynamoDB Amazon EMR Compute nodes AWS Data Pipeline

44 Amazon Redshift dramatically reduces I/O Column storage Data compression Zone maps ID Age State Amount CA WA FL WA 375 Direct-attached storage With row storage you do unnecessary I/O To get average age, you have to read everything

45 Amazon Redshift dramatically reduces I/O Column storage Data compression Zone maps ID Age State Amount CA WA FL WA 375 Direct-attached storage With column storage, you only read the data you need

46 Amazon Redshift dramatically reduces I/O Column storage Data compression Zone maps Direct-attached storage analyze compression listing; Table Column Encoding listing listid delta listing sellerid delta32k listing eventid delta32k listing dateid bytedict listing numtickets bytedict listing priceperticket delta32k listing totalprice mostly32 listing listtime raw COPY compresses automatically You can analyze and override More performance, less cost

47 Amazon Redshift dramatically reduces I/O Column storage Data compression Zone maps Direct-attached storage Track the minimum and maximum value for each block Skip over blocks that don t contain relevant data

48 Amazon Redshift dramatically reduces I/O Column storage Data compression DW.HS1.XL: 16 GB RAM 2 cores 2 TB disk DW.HS1.8XL: Zone maps Direct-attached storage 128 GB RAM 16 cores 16 TB disk > 2 GB/s scan rate Optimized for data processing High disk density

49 Amazon Redshift: start small and grow big Dense Storage Node (dw1.xlarge) 2 TB, 16 GB RAM, 2 cores Dense Compute Node (dw2.large) 0.16 TB, 16 GB RAM, 2 cores Single Node (2 TB) XL Cluster 2-32 Nodes (up to 64 TB) XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL Note: Nodes not to scale 8XL Dense Storage Node (dw1.8xlarge) 16 TB, 128 GB RAM, 16 cores, 10 GigE 8XL Dense Compute Node (dw2.8xlarge) 2.56 TB, 128 GB RAM, 16 cores, 10 GigE Cluster Nodes (up to 1.6 PB) 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL

50 How Amazon Redshift billing works Monthly bill = N = 3 nodes * 720 hrs * $0.25 = $540 dw2.large; Oregon; On-Demand 2 vcpus; 15 GiB RAM; 0.16 TB SSD Further details at

51 To sum up

52 Review: AWS Managed Database Services Amazon DynamoDB Amazon RDS Amazon ElastiCache Amazon Redshift Document and key-value store SQL database engines In-memory cache Data warehouse

53 Benefits of AWS Database Services Amazon S3 Amazon EC2 AWS Data Pipeline Amazon VPC Amazon SNS Amazon CloudWatch Managed services AWS handles installs, patching, restarts Easy to scale Grow as you need Pay only for what you use No up-front cost Designed for use with other AWS services

54 Thank You

Agenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache

Agenda. AWS Database Services Traditional vs AWS Data services model Amazon RDS Redshift DynamoDB ElastiCache Databases on AWS 2017 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon Web Services,