Lassoing the Clouds: Best Practices on AWS. Brian DeShong May 26, 2017

Size: px

Start display at page:

Download "Lassoing the Clouds: Best Practices on AWS. Brian DeShong May 26, 2017"

Ophelia Blankenship
5 years ago
Views:

1 Lassoing the Clouds: Best Practices on AWS Brian DeShong May 26, 2017

2 Who am I?

3 Agenda Running web servers Serving static content Security-related concerns Databases Logging

4 Regions + Availability Zones

5 Operating Web Servers

6 Amazon Machine Images (AMIs)

7 What is an Amazon Machine Image? Provides information required to launch an EC2 instance You specify the AMI to use when launching a new instance Amazon Linux by default CentOS, Ubuntu, Windows, etc. Some options on using AMIs

8 Just Enough OS Startup a bare instance, JeOS Install what you need at initial boot time User data : shell script that runs at initial boot Avoids AMI creation and maintenance Instances take longer to be ready for service

9 Just Enough OS User data script #!/bin/bash yum update -y web01 Amazon Linux yum install -y httpd24 php70 chkconfig httpd on groupadd www kernel sudo usermod -a -G www ec2-user chown -R root:www /var/www vim chmod 2775 /var/www openssl find /var/www -type d -exec chmod 2775 {} + openssh find /var/www -type f -exec chmod 0664 {} + // Download and put application code into place... service httpd start

10 Fully Baked AMI web01 kernel Amazon Linux web02 Amazon openssh Linux web03 sudo PHP 7.1 kernel Amazon openssh Linux vim ImageMagick web04 sudo PHP 7.1 kernel openssl Amazon openssh nginx Linux vim ImageMagick sudo PHP 7.1 kernel openssh openssl nginx vim ImageMagick sudo PHP 7.1 openssl nginx vim ImageMagick Added users, groups Filesystem setup Nginx starts at boot Places code on disk at boot Monitoring scripts, packages etc openssl nginx

11 Considerations How quickly do you need it in the event of a failure? or due to an increase in demand? Do you have the time and resources to maintain a fully baked AMI? My recommendation: Start with JeOS + configure at boot You can always create custom AMIs later, if needed

12 Fault Tolerance

13 No Single Point of Failure! Do not have a single point of failure! This can be a server, AZ, or even a region Always have at least two of everything If an EC2 instance dies, the other remains in service Holy grail: spread out across multiple regions

14 Use AZs Effectively us-east-1a us-east-1b us-east-1a us-east-1c web web web web web web web web web web db (write) db (read) web db (write) (standby) db (read) web db (read)

15 Leverage Auto Scaling Machine resources If CPU > 80% for 5 minutes, scale up web web web Threshold of unhealthy instances Want 5 web servers minimum? web web web If one dies, replace it Scale down If CPU < 50% for X minutes, scale down

16 Load Balancers

17 Don t use Sticky Sessions! By default, ELB sends request to instance with smallest load Sticky sessions pin your user to an instance behind ELB Uses a cookie to route the same client to a consistent target If instance fails, ELB stops routing to that instance; chooses another But you want to spread traffic around! As your pool of machines grows, the requests are balanced between them

18 Sticky Sessions UI

19 SSL Termination AWS Certificate Manager SSL Cert Load Balancer HTTP, port 80 HTTPS, port 443 web web web mod_ssl mod_ssl mod_ssl web web web mod_ssl mod_ssl mod_ssl

20 Use AWS Certificate Manager AWS Certificate Manager is amazing for managing SSL certificates, and free! Uses WHOIS contact information Automatically renews your certificate A single click, and it s renewed Updated everywhere it s used Can import external certificates, too SSL all the things!

21 Serving Static Content

22 Serve from static storage! Never serve static content from your web servers JavaScript, CSS, images, fonts, etc Don t use your computing resources Get the content to the end user as quickly as possible

23 AWS Simple Storage Service (S3) AWS s object storage service You pay by storage utilized, number of requests, and bandwidth S3 storage is made up of buckets of objects Perfect for storing static assets Store content at build time

24 S3 Standard Storage Class Standard - Infrequently Accessed Reduced Redundancy Storage

25 Use CloudFront

26 Security

27 Identity and Access Management Controls AWS services a user can access Which actions they can perform on those services Which resources are available Concepts of Users and Roles

28 Use IAM Roles on EC2 Instances Assign an IAM Role to an EC2 instance Enables you to obtain temporary access keys Can be used to access AWS resources AWS SDKs make requests with credentials from IAM Role No storing keys in your code base Much more flexible and maintainable

29 Security Groups Virtual firewall to control inbound and outbound traffic Typically attached to EC2 instances, load balancers, RDS instances Only allow traffic in on the necessary ports Restrict internal tool access to known IP addresses

30 Principle of Least Privilege Give users access to only the resources they need This applies to internal and external users Examples: Don t let an IAM role access every single S3 bucket! Specify each Don t allow every port in on a Security Group! Only what needs to be public

31 Relational Databases

32 AWS Relational Database Service Removes the usual maintenance associated with running databases Eases burden of software patches Backups / snapshots are incredibly convenient Can scale instances up and down in size

34 Operate in Multiple AZs us-east-1a us-east-1b Synchronous replication Primary Standby Patch: primary next Patch: standby first your-name.cluster-abc123.us-east-1.rds.amazonaws.com

35 Distribute read operations us-east-1a us-east-1b us-east-1c Writer Reader Reader us-east-1d us-east-1e Reader Reader your-name.cluster-ro-abc123.us-east-1.rds.amazonaws.com

36 AWS Aurora! Completely re-imagined storage of data Greatly reduces replica lag to single-digit milliseconds Read replicas launch in minutes Run MySQL or PostgreSQL engines on top

37 Logging

38 Centralize Application Logs Web servers (PHP error log, Apache logs) Cron jobs Asynchronous processes You need to be able to access these at any time

39 CloudWatch Logs CloudWatch Logs agent runs on EC2 instance Polls local log files on disk and copies to CW Logs Broken up into Log Groups and Log Streams Log Group: Apache access log, error log, PHP error log Log Stream: log entries from a specific instance

40 CW Logs Agent Install $ sudo yum install awslogs $ sudo service awslogs start [messages] file = /var/log/messages log_group_name = /var/log/messages log_stream_name = {instance_id} datetime_format = %b %d %H:%M:%S

41 CW Logs Console

42 Make Them Searchable Elasticsearch (Amazon ES) OSS utilities to tail Elasticsearch indexes Amazon ES includes Kibana Allows you to spot trends over time Dig through data for specific entries, time periods, etc.

43 Kibana

44 Proactively monitor and alert! Logs should really be empty day-to-day If they re not right now, fix that first CloudWatch Alerts for log entries over threshold Amazon Simple Notification Service: get paged, wake up! Develop as to avoid being woken up by pages

45 Recap

46 Operating Servers Choose an EC2 AMI strategy that suits your needs Don t have an SPOF Spread resources over AZs and/or Regions Keep SSL simple

47 Static Content Don t serve it from your web servers! Utilize S3 for all static content storage Leverage CloudFront for better global performance

48 Security Leverage IAM Roles to grant access to types of servers Limit Security Groups to only what s needed in and outbound Principle of Least Privilege is a great guide

49 Databases Again, spread across AZs Distribute read operations to slaves More sleep: automatic failover is a great asset

50 Logging Use CloudWatch Logs for central logging Don t just write the logs, monitor them! Alert on anomalies Find your bug and errors before your users do!

51 Thanks to our Sponsors!

52 Thanks! We re Hiring:

Lassoing the Clouds: Best Practices on AWS. Brian DeShong May 26, 2017

Lassoing the Clouds: Best Practices on AWS Brian DeShong May 26, 2017 Who am I? Who am I? Who am I? Who am I? Agenda Agenda Running web servers Agenda Running web servers Serving static content Agenda