Enhancing cloud applications by using messaging services IBM Corporation

Transcription

1 Enhancing cloud applications by using messaging services

2 After you complete this section, you should understand: Messaging use cases, benefits, and available APIs in the Message Hub service Message Hub messaging models and language support Worker offload pattern using Message Hub to make an application more responsive Delivery Assurance 2

3 Terminology Messaging: Technology that enables high-speed, asynchronous, program-to-program communication with reliable delivery. Topic: The channel through which apps publish messages and apps can subscribe to messages Queue: A channel through which apps "pull" messages they want to receive by using a PTP model instead of automatically receiving every message on a particular topic. Only one application gets the message in a queue. Sender, producer, or publisher: A program that sends a message by writing the message to a channel. Receiver, consumer, or subscriber: A program that receives a message by reading (and deleting) it from a channel. PTP: Point to Point Messaging (uses queues). Messaging Pub/Sub: Publish/Subscribe (uses topics). Sender or producer Msg Queue or topic Msg Receiver or consumer 3

4 Trends in cloud application architecture Data center architecture Cloud-centric architecture HTML + Client JavaScript HTML + Client JavaScript Business Logic Microservices Data Interaction through REST and lightweight asynchronous messaging. Data Data 4 Test, deploy, and operate as a whole Often scale monolithic app Powerful integration runtime Homogeneous technologies Caveat: oversimplification Small empowered DevOps teams own each service Test, deploy, monitor, operate individual services Polyglot languages and persistence Services scale automatically as needed

5 IBM Message Hub in Bluemix PaaS 5

6 Message Hub Message Hub Use cases for Message Hub Worker offload Intensive work offloaded and distributed among worker processes to be performed asynchronously Processing images or videos Performing text analytics Event driven Take one or more actions when something interesting happens logs and update dashboards when a build finishes Upload videos after finishing transcoding 6

7 Message Hub Message Hub More use cases Delayed processing Schedule a task to happen at a specific time Run in detailed reports when app use is low Generate end of day summary Third-party integration Ensure applications remain responsive even when third-party systems are not available or responding fast enough Update an existing CRM system Book an appointment 7

8 Message Hub APIs Kafka API: Native API for accessing Message Hub Low latency and higher throughput More features than the Kafka REST API Kafka REST API: RESTful interface to Message Hub Quickly get started using Message Hub Cases where low throughout use cases where latency is not a critical factor MQLight API: AMQP-based messaging API for Java, Node.js, Python, and Ruby Message Hub clients Higher level of abstraction than the Kafka API or the Kafka REST API Supports the use of hierarchy and wildcards in topic structure Includes additional support for fault tolerance and QoS enabled by creating a Message Hub topic: "MQLight" Access Message Hub using credentials from VCAP_SERVICES 8

9 Message Hub messaging model: send messages MQ Light API Topic address space Sender application 1. Send ( test/a, Hello ); 2. Send ( test/a, World! ); Kafka API Topic address space Producer application 1. Send ( testa, Hello ); 2. Send ( testa, World! ); 9

10 Message Hub messaging model: simple receive MQ Light API Sender application Topic address space DESTINATION Pattern=test/a Receiver application 1. Hello 2. World! 1. Send ( test/a, Hello ); 2. Send ( test/a, World! ); Kafka API Producer application Topic address space Queuing or Pub Sub Model Topic name=testa Consumer application 1. Hello 2. World! 1. Send ( testa, Hello ); 2. Send ( testa, World! ); 10

11 MQ Light API: Pub/sub topic hierarchies and wildcards MQ Light API supports multilevel topics and wildcards for Pub/Sub Example of a topic hierarchy USA USA/Texas USA/Texas/Austin USA/Texas/Houston Wildcards Multilevel wildcard: # Matches zero or more levels USA/# matches all 4 topics shown above USA/Texas/# matches the last 3 Single level wildcard: + Matches one and only one topic level USA/+ matches USA/Texas only USA/Texas/+ matches USA/Texas/Austin and USA/Texas/Houston 11

12 Message Hub messaging model: pub/sub MQ Light API Sender application 1. Send ( test/a, Hello ); 2. Send ( test/a, World! ); Topic address space DESTINATION Pattern=test/a DESTINATION Pattern=test/# Receiver applications Client 1 Client 2 1. Hello 2. World! 1. Hello 2. World! Kafka API Producer application 1. Send ( testa, Hello ); 2. Send ( testa, World! ); Topic address space Pub Sub Model Topic name=testa Pub Sub Model Topic name=testa Consumer applications Client 1 Client 2 1. Hello 2. World! 1. Hello 2. World! 12

13 Message Hub messaging model: persistent destinations MQ Light API Sender application Topic address space Hello World! DESTINATION Pattern=test/a Receiver application Disconnected client 1. Send ( test/a, Hello ); 2. Send ( test/a, World! ); Persistence duration called Time to Live Kafka API Topic address space Hello World! Consumer application Disconnected client Producer application 1. Send ( testa, Hello ); 2. Send ( testa, World! ); Queuing or Pub Sub Model Topic name=testa Persistence duration called Log retention 13

14 Message Hub messaging model: sharing MQ Light API Topic address space Sender application Receiver application Client 2 1. Hello 1. Send ( test/a, Hello ); 2. Send ( test/a, World! ); DESTINATION Pattern=test/# Share=myshare SHARING Client 3 2. World! 14

15 Message Hub messaging model: sharing Kafka API Topic with 2 Partitions Producer Application A Consumer Application B Queuing Model Hello 1. Send ( testk, Hello ); 2. Send ( testk, World! ); topic= testk consumer group= myshare Consumer Application C World! 15

16 Worker offload pattern using Message Hub Improve the level of responsiveness by offloading work from the application's main event loop. 17

17 MQ Light time to live and delivery assurance Time to Live Destination based Messages stored at the destination for specified time if not retrieved On expiry, all held messages will be discarded and no new messages will be stored Message based Message stored at the destination for specified time if not retrieved On expiry, message will be discarded Message delivery model At most once delivery (QoS 0) Appropriate when occasional loss of data caused by a communication disruption is acceptable At least once delivery (QoS 1) Required when there can be no loss of data Duplicate messages are possible and should be discarded programmatically

18 Related links IBM Bluemix documentation: Getting started with Message Hub IBM Bluemix documentation: Kafka API IBM developerworks article: Kafka REST API IBM developerworks article: MQ Light API IBM developerworks article: Microservices in Bluemix Video Message Hub: Getting Started with IBM s Kafka in the Cloud 19