SQS, SWF, and SNS 7/24/17. References. Amazon Simple Queue Service(SQS)

Transcription

1 SQS, SWF, and SNS Chapter 8 References All informa6on in this presenta6on was obtained from the following sources with all credit due to the listed authors: J. Baron, H. Baz, T. Bixler, B. Gaut, K. E. Kelly, S. Senior, and J. Stamper, AWS Cer(fied Solu(ons Architect official study guide: Associate exam. United States: Sybex, a Wiley brand, AWS DocumentaHon, Amazon Web Services, Inc. [Online]. Available: hops://aws.amazon.com/documentahon/. [Accessed: 15- Jul-2017]. Amazon Simple Queue Service(SQS) Amazon SQS Queue- a buffer between the applica6on components that receive data and those that process data within your system. Fully managed message queuing service. Fast, reliable, scalable, simple, and cost effec6ve. Transmits any volume of data without losing messages or requiring other services to be con6nuously available. Ensures message delivery at least once and supports mul1ple readers and writers interac6ng with the same queue. Service is reliable and efficient but does not guarantee first in, first out(fifo) delivery. 1

2 Message Lifecycle Delay Queues & Visibility Timeouts Delay queues allow you to postpone delivery of queued messages, which can be set between seconds (Up to 15 minutes). Queues can be set by using CreateQueue and seung the DelaySeconds avribute(defaulted at 0 seconds). Visibility Timeouts are similar to delay queues, however, the delay queue hides the message when it is added to the queue, and visibility 6meouts hide the message axer it is pulled from the queue(up to 12 hours). A message is considered in flight if neither of the above are applied (120,000 in flight max). Queue Operations, IDs, IdentiHiers, & Attributes Opera6ons only available to account owner or iden6ty with granted permissions. SQS assigns new queues with a queue URL. Each message is assigned a unique ID(100 char max), which is returned when the message is sent. SQS also supports message avributes, which allow you to avach metadata to your messages (10 avributes per message). Metadata can include 6mestamps, signatures, etc. Opera6ons: ListQueues DeleteQueue SendMessage SendMessageBatch ReceiveMessage DeleteMessage DeleteMessageBatch PurgeQueue ChangeMessageVisibility ChangeMessageVisibilityBatch SetQueueAVributes GetQueueAVributes GetQueueUrl ListDeadLeVerSourceQueues AddPermission RemovePermission 2

3 Long Polling & Dead Letter Queues Long polling allows you to send a WaitTimeSeconds argument to ReceiveMessage(Up to 20 seconds). Reduces the load on the client by returning a message immediately or wai6ng the specified 6me then returning regardless of message s existence. Dead leder queues are queues that source queues can target to send messages that could not be processed successfully. Allows you to sideline unsuccessful messages for analysis. Amazon Simple WorkHlow Service(SWF) Amazon SWF makes it easy to build applica6ons that coordinate and work across distributed components. Workers perform tasks, which can run on a cloud infrastructure or your own premises. SWF stores tasks, assigns them to workers when they are ready, monitor their progress, and maintains their state. With SWF you can implement, deploy, scale, and modify these applica6on components independently. WorkHlows, Domains, & History Workflows are asynchronous applica6ons within SWF that coordinate and manage the execu6on of ac6vi6es that can be run across mul6ple compu6ng devices that feature either sequen6al of parallel processing. Workflow domains help to scope SWF resources within your AWS account. Domains must be specified for all the components of a workflow. You can have more than one workflow in a domain, however, workflows in different domains cannot interact with one another. All workflows have a history, which is a detailed complete and consistent record of every event that occurred since the execu6on of the workflow. 3

4 Actors, Tasks, & Task Lists SWF consists of a number of programma6c features known as actors, which can be workflow starters, ac6vity starters, or deciders. A workflow starter is any applica6on that can ini6ate workflow execu6on. EX: A mobile applica6on where a customer orders food or taxi service. The logic that coordinates tasks in a workflow, which can be run sequen6ally, in parallel, synchronously, or asynchronously, is called a decider. An ac1vity worker is a single computer process(or thread) that performs tasks in your workflow. Tasks are work assignments and can be ac1vity tasks, AWS Lambda tasks, or decision tasks An ac6vity task tells a worker to perform it s func6on. An AWS Lambda task executes AWS Lambda func6ons instead of regular tasks. A decision task tells a decider the state of the workflow execu6on has changed so the decider can evaluate the next steps to be taken. Tasks can be organized into task lists, which are flexible mechanisms to route tasks to workers. More on WorkHlows Deciders and ac6vity workers communicate with SWF using long polling. SWF objects are uniquely iden6fied by workflow type, ac6vity type, decision/ac6vity tasks, and workflow execu6on. Registered workflow and ac6vity types are iden6fied by their domain, name, and version, which can be specified using RegisterWorkflowType and RegisterAc1vityType, respeclully. A single execu6on of is iden6fied by the domain, workflow ID, and run ID. A workflow is open axer it is started. An open workflow can be closed as completed, cancelled, failed, or 6med out. Lifecycle of a WorkHlow Execution There are 20 steps to a workflow execu6on. 4

5 Amazon Simple NotiHication Service(SNS) Amazon SNS is a web service for mobile and enterprise messaging that enables you to set up, operate, and send no6fica6ons. Follows a publish-subscribe(pub-sub) messaging paradigm. No6fica6ons are delivered to clients using a push mechanism, which eliminates the need to poll for new informa6on and updates. Can send no6fica6ons to Apple, Android, Fire OS, & Windows devices. Two types of clients, publishers and subscribers(producers and consumers). Publishers communicate to subscribers asynchronously by sending a message to a topic. Messages sent to topics are automa6cally forwarded to each subscriber. Messages can be sent using different methods such as Amazon SQS, HTTP, HTTPS, , SMS, and AWS Lambda. Common SNS Scenarios Fanout scenarios happen when an Amazon SNS message is sent and replicated and pushed to mul6ple SQS queues, HTTP endpoints, or addresses, which allows for parallel asynchronous processing. Applica1on/System alerts are SMS or no6fica6ons that are triggered by predefined thresholds. You can receive no6fica6ons if events happen within an AWS service. Push & Text Messaging can help transmit individual or group messages via or SMS. Useful for pushing relevant topics to subscribers. Mobile Push No1fica1ons allow you to send messages directly to mobile applica6ons. Useful for upda6ng. Recap Amazon SQS is a unique service designed to help you decouple your infrastructure. SQS allows you to store messages on a reliable and scalable infrastructure as they travel between distributed components of your applica6ons that perform different tasks. Amazon SWF allows you to create applica6ons that coordinate work across distributed components. SWF is driven by tasks, which can be managed, keeping in mind, inter-task dependencies, scheduling, and concurrence. Workflows allow you to have full control over implemen6ng tasks without worrying about underlying complexi6es. Amazon SNS allows you to push individual or mul6ple messages to a large number of recipients. SNS supports a wide variety of needs, including monitoring applica6ons, workflow systems, 6me-sensi6ve updates, and any other applica6on that generate or consumes no6fica6ons. 5