Create High Performance, Massively Scalable Messaging Solutions with Apache ActiveBlaze

Transcription

1 Create High Performance, Massively Scalable Messaging Solutions with Apache ActiveBlaze Rob Davies Director of Open Source Product Development, Progress: FuseSource - Rob Davies Director of Product Development PROGRESS: FuseSource rodavies@progress.com Apache ActiveMQ, ServiceMix and Camel PMC member Co-founder of ActiveMQ, ServiceMix, Camel, ActiveBlaze Co-Author of ActiveMQ in Action What this Session will Cover The need for configurable low latency, scalable peer to peer messaging The messaging model of Apache ActiveBlaze Reliable multicast Reliable point-cast Group membership Clustering framework Using JMS with Apache ActiveBlaze What you will learn How to use ActiveBlaze for collaborative low latency applications The reliable multicast protocols available, how they work and when to use them Using ActiveBlaze for true point-2-point messaging Maintaining distributed group membership Using ActiveBlaze for cluster state management Using JMS with ActiveBlaze Why Apache ActiveBlaze? Hub-less Messaging Faster publish/subcribe Group collaboration State membership members can dynamically join and leave a group True peer to peer message interactions Queue style messaging (one peer selected to process a message Cluster State Support cluster shared-state Locking State and lock failover without the need for a hub Messaging 101 Messaging is Loosely coupled exchange of messages between producers and consumers So producers and consumers know nothing of each other they only know about destinations (queues and topics) The ideal approach to building high performance distributed systems Web services done right is really just a form of messaging using pointy brackets Can be persistent or non-persistent May add timeouts & priorities to messages Can be transactional Several receive and send operations can be performed as a unit of work.

2 Topics (publish & subscribe) One message goes to 0-to-many consumers based on the current subscribers Think like mailing lists or discussion forums The producer is decoupled from the consumers; it doesn t need to know who all the consumers are Ideal for publishing business events Distributed observer pattern Allows one part of your system to notify anyone else who may be interested in an event Queues (load balancing) Messages are load balanced across many consumers Each message goes to exactly one consumer Consumers compete for messages Its easy to browse and monitor queues Monitor the performance of your application, find the hotspots, alert if queues are too full If a consumer crashes during the processing of a message it is automatically redelivered to another consumer So queues implement reliable load balancing with optional persistence Queues - ideal for grid style Ideal for building reliable grid applications Fire requests at a cluster of servers Let the JMS provider provide load balancing, redelivery and optional persistence If the queue starts filling up, just boot up new servers Easily monitor the performance and status of the system View the throughput rates on each queue and queue size Deals easily with parts of your system being taken down for maintenance Broadcast Broadcast on Topic Wildcard publishing Wildcard subscribing Use selectors Pluggable broadcast Pluggable Reliability Setting up a Broadcast Channel Subscribing to messages import org.apache.activeblaze.*;... BlazeChannelFactory factory = new BlazeChannelFactory(); BlazeChannel sender = factory.createchannel(); //start the channel and send a message sender.start(); String destination = "foo.bar.*"; BlazeMessage msg = new BlazeMessage("test payload"); sender.broadcast(destination,msg); BlazeChannel receiver = factory.createchannel(); receiver.start(); //add a listener String destination = "foo.bar.cheese"; receiver.addblazetopicmessagelistener(destination, new BlazeMessageListener() { public void onmessage(blazemessage msg) { System.out.println("Got a msg: " + msg); );

3 Configurable transport stack A channel processes messages before dispatch Messages are compressed if larger than a certain limit Messages are fragmented if larger than a certain size after compression Messages can be encrypted A channel processes messages on receipt Checks for duplicates Asks for retransmission Orders messages BlazeMessage A BlazeMessage default message for transportation Implements a Map<String,Obect> of key/value pairs Values can be primitives, byte arrays, Strings or Serializable Objects Supports recursion a Value can be another Map Uses ActiveMQ-protobuf extension to google s protobuf -for crosslanguage support. Different types of Broadcast Pluggable Reliable Multicast Flow control Nack Based Totem SRM Layered Multicast Different types of Broadcast Pointcast for where you can t use multicast SWP UDP TCP Group Membership Members advertize themselves Multiple group membership Configurable heartbeat Location advertized Member lifecycle Inbox Messaging Group Membership for Point-cast Members advertize themselves to a location service Location service only used for membership info Multiple locators can be used Communication still p-2-p

4 BlazeGroupChannel Provides membership state information Supports Inboxes can communicate directly to another member of the group Supports Queue semantics and reliable delivery Supports Recorded delivery local persistence for messages Setting up a GroupChannel BlazeGroupChannelFactory factory = new BlazeGroupChannelFactory(); BlazeGroupChannel channel = factory.creategroupchannel("my Channel"); //join a group channel.addtogroup("test"); channel.start(); //listen for other members channel.addmemberchangedlistener(new MemberChangedListener(){ public void memberstarted(member member) { System.out.println("Member started"); public void memberstopped(member member) { System.out.println("Member stopped"); ); Supporting Queues GroupChannel supports Queue Messaging Routing done by the publisher Reliable Round-robin dispatch True p-2-p Sending to a Queue import org.apache.activeblaze.*;... BlazeGroupChannelFactory factory = new BlazeGroupChannelFactory(); BlazeGroupChannel sender = factory.createchannel(); //start the channel and send a message sender.start(); String destination = "foo.bar.*"; BlazeMessage msg = new BlazeMessage("test payload"); sender.send(destination,msg); Receiving messages from a Queue BlazeGroupChannel receiver = factory.createchannel(); receiver.start(); //add a listener String destination = "foo.bar.cheese"; receiver.addblazequeuemessagelistener(destination, new BlazeMessageListener() { public void onmessage(blazemessage msg) { System.out.println("Got a msg: " + msg); Recorded Delivery Consumers do not need to be available at the time messages are published Storing messages locally to the publisher improves performance Clients optimized to do as much work as possible asynchronously. Frees up your application threads do get more work done. High performance journaling message store Messages are quickly and reliably stored on disk using write batching techniques. );

5 Mapping JMS to ActiveBlaze JMS is the de-facto Messaging API for Java JMS 1.1 support Map a JMS Connection to BlazeGroupChannel Session based transactions JMS Message Type support JMS optimizations NON_PERSISTENT DeliveryMode by default Acknowledgements for persistence only Straight-throw message processing No dispatch thread per Session No context switching Faster throughput JMS selectors Performance X3 Performance for Topics X2 Performance for Queues Better scalability for Topics Faster failover for Queues Using ActiveBlaze for cluster management Hubless state management Elected Read/Write lock manager Lock manager failover support Dynamic cluster membership Cluster listeners Cluster members can use ActiveBlaze for inter-cluster communications How Cluster Management works Questions?