Scaling on AWS. From 1 to 10 Million Users. Matthias Jung, Solutions Architect

Transcription

1 Berlin 2015

2 Scaling on AWS From 1 to 10 Million Users Matthias Jung, Solutions Architect

3 How to Scale?

4

5

6 lot of results not the right starting point

7 What is the right starting point?

8 First some basics

9 AWS Regions US-WEST (Oregon) AWS GovCloud (US) EU-WEST (Ireland) CHINA (Beijing) ASIA PAC (Tokyo) US-EAST (Virginia) US-WEST (N. California) ASIA PAC (Sydney) SOUTH AMERICA (Sao Paulo) ASIA PAC (Singapore)

10 Availability Zones (AZs) US-WEST (Oregon) AWS GovCloud (US) EU-WEST (Ireland) CHINA (Beijing) EU-CENTRAL (Frankfurt) ASIA PAC (Tokyo) CHINA (Beijing) US-EAST (Virginia) US-WEST (N. California) ASIA PAC (Sydney) SOUTH AMERICA (Sao Paulo) ASIA PAC ASIA (Singapore) PAC (Singapore)

11 Edge Locations

12 Services Deployment & Administration Application Services Compute Storage Database Networking AWS Global Infrastructure

13 Services AWS OpsWorks CloudWatch Elastic Beanstalk AWS IAM AWS CloudFormation AWS Data Pipeline AWS CloudTrail SNS CloudSearch SQS Deployment & Administration DynamoDB ElastiCache RDS SES SWF Elastic Transcoder Application Services RedShift Compute Storage Database S3 EC2 EMR VPC Networking AWS Storage Gateway Glacier CloudFront Kinesis Route 53 AWS Direct Connect AWS Global Infrastructure

14 1

15 Day 1, User 1 User Route 53 Complete stack on single EC2 Instance Single Elastic IP Address Route 53 for DNS EC2 Instance Elastic IP Address

16 We need a bigger box Change instance size Change instance family Increase EBS PIOPS i2.4xlarge m3.xlarge t2.small

17 First steps User Route 53 Quite Simple Scales up to the thousands Elastic IP Address EC2 Instance

18 First steps User Route 53 Will hit an endpoint eventually No failover, no redundancy All eggs in one basket EC2 Instance Elastic IP Address

19 1,000

20 1000 Users and more User Route 53 Separate database and app Elastic IP Address Managed database service? Web Instance Database Instance

21 Database Options Self-Managed Fully-Managed Database Server on EC2 RDS DynamoDB Redshift Choice of Software and Version Bring your own license (BYOL) MS SQL, Oracle, Postgre & MySQL as managed service License included or BYOL Managed NoSQLservice with SSD storage Seamless scalability Zero administration Data Ware House as a service (SQL) Massively parallel High scalability Fast access

22 Which database technology to start with?

23 Why a SQL database? Established and well worn technology Lots of existing code, tools, communities, books Clear patterns to scalability You aren t going to break SQL DBs in your first 10 million users. No really, you won t* *Unless you are doing something SUPER weird with the data or MASSIVE amounts of it and even then SQL will have a place in your stack

24 When is NoSQL the better fit? Huge amounts of data (Terra Bytes) Thousands of write/update operations per second Applications with high latency requirements Unstructured data, no fix tables Data without or very loose relationships Storing meta data Expertise already in the team

25 DynamoDB Fully Managed Fast and predictable performance Fully distributed and fault-tolerant architecture Provisioned throughput Predictable performance Strongly consistent reads Fault tolerance built in Monitoring built in Security built in (IAM support) Integration with AWS Big Data Services

26 1000 Users and more Separate database and app Managed database: RDS User Elastic IP Address Route 53 Web Instance RDS DB Instance

27 10,000

28 10,000 Users and more User Route 53 Failover & Redundancy Multiple Availability Zones RDS Multi-AZ Elastic Load Balancing Web Instance Elastic Load Balancing Web Instance RDS DB Instance Active (multi AZ) Availability Zone A RDS DB Instance Standby (Multi-AZ) Availability Zone B

29 Elastic Load Balancing Designed for fault-tolerant and highly scalable applications Elastic Load Balancing Highly available and elastic Health checks Layer 4 and 7 support SSL termination Monitoring built in Access logs IPv6 support

30 Horizontal Scaling User Route 53 Elastic Load Balancing Web Instance Web Instance Web Instance Web Instance Web Instance Web- Instance Web- Instance Web- Instance RDS DB Instance Read Replica RDS DB Instance Read Replica RDS DB Instance Master (Multi-AZ) RDS DB Instance Standby (Multi-AZ) RDS DB Instance Read Replica RDS DB Instance Read Replica Availability Zone A Availability Zone B

31 100,000

32 Shift some load around User Route 53 CloudFront Move static content to S3 Deliver content via CloudFront Elastic Load Balancing Cache DB queries in ElastiCache Move session state to ElastiCache or DynamoDB Web Instance ElastiCache S3 RDS DB Instance Master (Multi-AZ) DynamoDB Availability Zone

33

34 November traffic to.com November

35 November traffic to.com Provisioned Capacity 76% November 24%

36 November traffic to.com November

37 Auto Scaling Triggers Auto-Scaling Policy Automatically adapts capacity to demand Integration with CloudWatch Integration with Elastic Load Balancing For scaling and availability CloudWatch as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones us-east-1a --min-size 4 --max-size 200

38 100,000 users + User Route 53 CloudFront Elastic Load Balancing S3 Web Instance Web Instance Web Instance Web Instance Web Instance Web Instance RDS DB Instance Master (Multi-AZ) RDS DB Instance Read Replica ElastiCache RDS DB Instance Standby (Multi-AZ) RDS DB Instance Read Replica ElastiCache DynamoDB Availability Zone Availability Zone

39 1,000,000

40 Loose coupling sets you free Decoupling is a prerequisite to scale and optimize Independent components Design everything as blackbox Decouple interactions Clean interfaces

41 Decoupling in action EC2 Instance Loose coupling Upload photo Resize photo

42 Decoupling in action EC2 Instance Loose coupling Upload photo Resize photo Q SQS Upload photo Resize photo EC2 Instances

43 Decoupling in action EC2 Instances Loose coupling Upload photo Upload photo Upload photo Resize photo Resize photo Resize photo Q SQS Upload photo Resize photo Resize photo Resize photo EC2 Instances

44 Think services Fine-granular services instead monoliths Consistent and coherent services with specific responsibilities 100% independent services Services communicate via welldefined APIs only

45 Think services Fine-granular services instead monoliths Consistent and coherent services with specific responsibilities 100% independent services Services communicate via welldefined APIs only

46 Think services Fine-granular services instead monoliths Consistent and coherent services with specific responsibilities 100% independent services Services communicate via welldefined APIs only = principle behind AWS und.com

47 Don t reinvent the wheel If you find yourself writing your own Notification system component Search engine SNS CloudSearch SQS Workflow engine Queue Transcoding system Monitoring system SES SWF Elastic Transcoder

48 Don t reinvent the wheel If you find yourself writing your own Notification system component Search engine SNS CloudSearch SQS Workflow engine Queue Transcoding system Monitoring system SES SWF Elastic Transcoder take a deep breath and stop it now!

49 1 Mio users and more User Route 53 CloudFron t Elastic Load Balancing Web Instance Web- Instance Web Instance Web Instance Worker Instance Worker Instance SQS S3 ElastiCache DynamoDB RDS DB Instance Read Replica RDS DB Instance Read Replica Verfügbarkeitszone RDS DB Instance Master (Multi-AZ) Internal App Server Internal App Server CloudWatch SES

50 10,000,000

51

52 SERVER METRICS AGGREGATED METRICS LOG ANALYSIS EXTERNAL MONITORING

53 AWS Marketplace & Partners Customer can find, research, buy software Simple on demand pricing Launch in minutes Billing integrated into your AWS account products across 20+ categories Learn more at: aws.amazon.com/marketplace

54

55 Automation AWS Elastic Beanstalk AWS OpsWorks AWS CloudFormation EC2 Convenience Control

56 Scaling the database Federation: distribute database structure to different database systems by function Sharding: distribute data to different database systems (e.g. users by region) NoSQL: offload database by moving certain workloads to NoSQL databases

57 and this leads us to 10 million users

58 In a nutshell: scaling with AWS to 10 mio users Distribute infrastructure across AZs Caching, caching, caching Decoupling and think services Don t reinvent the wheel Auto-Scaling (once you have done your homework) Monitoring on all levels Automate deployment and operation

59 100,000,000

60 Million Users Iterate on previous patterns More fine-granular services More monitoring, fine-tuning and optimization From multi-az to multi-region More and more individual solutions

61 Some reading aws.amazon.com/documentation aws.amazon.com/architecture aws.amazon.com/start-ups

62 Thank you!

63 Web Foodpanda Foodpanda GmbH Mathias Nitzsche, CTO

64 Online food ordering platform Active in >40 emerging markets

65 Mid 2012: Launch DevOps Test of business model in few example markets (SG, IN) Small IT team with very limited resources Basic setup: Route53, ELB, EC2, RDS, CloudFront AWS: Quick setup; easy to use; no long term contract; standardized; documented

66 2013: Global Expansion DevOps 1-2 country launches per week AWS: Global coverage; flexibility; pricing model

67 2014: Rapid Growth DevOps Architecture gradually changes towards microservices VPC, Autoscaling, CloudFormation, SQS, SNS, S3, ElastiCache AWS: scalability; extensibility; openness; security; automatization; high availability using Multi-AZ

68 2015: Market leadership DevOps Ongoing growth + Acquisitions, huge TV campaigns, additional business models ( ) AWS: Buys time Foodpanda AWS Costs

69 Instances Throughput Asia with 13 countries, June 28th RestaurantBackend Frontend Backend 04:00 08:00 12:00 16:00 20:00 0:00

70 Cost of scalability Microservices, VPC, nosql, Scalability

71 Current Challenges Ongoing cost and performance optimization Spot / reserved / proper sized instances; merging regions; improving the app) Deployment with prepared AMIs Monitoring (CloudWatch, Icinga, Kibana, NewRelic, Pings) Security review

72 What s next on our wishlist? New regions in India & Russia? Microservice hosting? Mature enough for the Chaos monkey? Elastic File System? Aurora? A little more love for AWS Route53 or AWS CloudFront?...and maybe even basic DDoS?

73 Thank you!