Additional AWS Services

Transcription

1 Additional AWS Services

2 Kinesis Streams Enables you to build custom applications that process or analyze streaming data for specialized needs. It can continuously capture and store TB of data per hour from thousands of sources such as website clickstreams, financial transactions, social media feeds, IT logs, and location-tracking events.

3 E C 2 Instances E C 2 Instance A m a z o n S3 Shard 1 Shard 2 C lient E C 2 Instance D y n a m o D B Shard N Mobile Client E C 2 Instance Amaz o n R edshift Traditio nal Server A m a z o n Kinesis Stream E C 2 Instance A m a z o n E M R Producers Consumers

4 By default data is stored for 24 hours, but can be increased to 7 days

5 Streams Terminology P ro ducers D ata records S h a rd s C onsumers S e q u e n c e number Partition ke y Data b l o b

6 Producers A m a z o n Kinesis Streams A P I A m a z o n Kinesis Producer Library ( K P L ) A m a z o n Kinesis A g e n t

7 A uniquely identified g r o u p of data records in a stream A stream is c o m p o s e d of one or more shards, e a c h of which provides a fixed unit of capacity Shards C a n support up to 5 transactions per s e cond for reads Max total data read rate of 2 MB/s U p to 1,000 records per s e cond for writes Max total data write rate of 1 MB/s (including partition keys)

8 f your data rate increases, add more shards to increase the size of your stream. Remove shards if the data rate decreases.

9 Used to g r o u p data by shard within a stream Partition Keys Stream service segre gate s data records b e l o n g i n g to a stream into multiple shards Use partition keys associated with e ach data record to determine which shard a give n data record b e l o n g s to Specified by the applications putting the data into a stream

10 Sequence Number E a c h data record has a unique sequence number A s s i g n e d b y Streams after you write to the stream with client.putrecords or c lie nt.putre cord

11 Data Blobs The data your producer adds to a stream. The maximum size of a data blob (the data payload after Base64- decoding) is 1 megabyte (MB).

12 Consumers Consumers get records from Amazon Kinesis Streams and process them. These consumers are known as Amazon Kinesis Streams Applications.

13 AWS CloudFormation Gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.

14 Supported Services Virtual Private C l o u d ( V P C ) A u t o S c a l i n g Elastic C o m p u t e C l o u d ( E C 2 ) Elastic L o a d Balancer ( E L B ) Identity a n d A c c e s s Management (IAM) Route 53 A m a z o n S3 C l oudwatch Relational D a t a b a se Service D y n a m o D B C l oudfront CloudTrail Elastic Beanstalk A m a z o n ElastiCache S i m p l e Notification Service ( S N S ) S i m p l e Q u e u e Service ( S Q S ) A m a z o n Kinesis A W S O p s Wo r k s A m a z o n Redshift A m a z o n S i m p l e D B

15 Templates and Stacks Templates Templates are architectural d e s i g n s Yo u can create, update a n d delete templates S ta c ks S ta c ks are d eployed resources b a s e d on templates You can create, update and delete stacks u sing templates CloudFormat ion templates are written in J S O N

16 Youdon t need to figure out the order for provisioning A W S services Yo u don t need to worry about making d e p e n d e n cies work Templates Modify a n d update templates in a controlled a n d predictable way - In effect applying version control Visualize your templates as d i a gra m s a n d edit them using a dra g - a n d - d rop interface with the A W S C l oudformation Designer

17 Deploying Stacks A W S Management C onsole C o m m a n d L i n e Interface A P I s

18 Template Elements File format a n d version List of resources and associated configuration values Template p aram eters Output values List of data tables

19 AWS Elastic Beanstalk A service for deploying and scaling web applications and services. Upload your code and Elastic Beanstalk automatically handles the deployment, from capacity provisioning, load balancing, auto-scaling to application health monitoring.

20 Integrates with V P C Integrates with IAM C a n provision R D S instances Elastic B eanstalk Overview Full control of resources C o d e is stored in S3 Multiple environments are supported to e nable ve rsioning C h a n g e s from Git repositories are replicated Linux a n d W i n d ows 2008 R2 AMI support

21 D e p l oy code using a W A R file or Git re pository Elastic B eanstalk Overview Use A W S toolkit for Visual Studio and A W S Toolkit for Eclipse to d e p l oy to Elastic Beanstalk Elastic BeanStalk is fault tolerant within a single region (not FT between regions) B y default your applications are publicly accessible

22 Elastic Beanstalk Management C l oudwatc h monitoring Adjust Application Server settings R u n other application components A c c e s s l o g files without l o g g i n g into application ser vers

23 AWS OpsWorks A configuration management service that helps you automate operational tasks like software configurations, package installations, database setups, server scaling, and code deployment using Chef.

24 Automation platform that transforms infrastructure into c o d e What Is Chef? Automates how applications are configured, deployed, a n d m a n a g e d across your network C h e f server stores your recipes a n d configuration data C h e f client ( n o d e ) is installed on e a c h server

25 Use the A W S Management C onsole Consists of two elements: S t a c k a n d L a y e r s OpsWorks Com ponents S ta c ks are containers of resources ( E C 2, R D S, E L B ) that you want to m a n a ge collectively Eve r y S t a c k contains one or more layers: - Web application layer - Database layer L ayers automate the deployment of p a c ka ge s for you

26 Summary Kinesis Streams C l oudformation Elastic Beanstalk O p s Wo r k s