WebSphere Default Messaging

Transcription

1 SHARE Summer 2009 Denver, CO August Session 1176 WebSphere Default Messaging Concepts common across all WebSphere platforms, but we show z/os here Don Bagwell IBM Washington Systems Center 1

2 Session Objective and Agenda Purpose is to introduce default messaging and provide an understanding of how it works and how it relates / works with WebSphere MQ Messaging Introduction Set the stage with some key concepts and terminology related to messaging Java Messaging Service (JMS) An introduction to this key element of messaging in a Java EE standardized world JMS and JMS Providers Exploring what options you have when configuring the actual messaging infrastucture The Service Integration Bus (SIBus) This is what implements the default messaging provider Interoperating with WebSphere MQ For those who have MQ and want to continue using it in conjunction with WebSphere Administration and Creating the SIBus and Destinations A quick tour of the panels and the process Basics 2 IBM Americas Advanced Technical Support The agenda for this presentation. 2

3 Messaging Basics 3 IBM Americas Advanced Technical Support 3

4 Introduction to Messaging The concept has been around for a long time. It involves turning message over to messaging system, and then having other side pick it up: Program 1 Interface Intermediary function to store and forward messages 2 Msg Queue Interface 3 Program 1. Sender interacts with interface and puts message 2. Messaging system takes message and holds it, waiting for receiver to get it 3. Receiver interacts with interface to get message Chief advantage to this is it avoids complicated handshake protocols typical of synchronous communications. Also helps facilitate any to any platform communication all that s needed is access to messaging and agreed to message format. WebSphere MQ 4 IBM Americas Advanced Technical Support For those who might not have a background in messaging, it is a method of exchanging information between two application programs in a way that is decoupled by a middle point where the information -- the message -- is momentarily stored. Compare this to the other method of interprogram communication where some kind of real-time session is established, and the communication between the two parties that takes place is done according to a set of protocols agreed to ahead of time. The human analogy would be a telephone call vs. . A telephone call requires the other party to be there, and some sense of courtesy about who talks and who doesn t at any given time. During the call a session exists between the callers. on the other hand does not require the other party to be there -- the goes to an intermediary holding spot (the mail server) where it waits for the other side to pick it up. The concept of messaging has been around for a while now, with IBM s most famous messaging product being WebSphere MQ Series, which we ll talk about quite a bit in this presentation. Messaging has become so popular because it is far less complicated to do than a real-time, synchronous session between the two parties. With messaging all you need is a connection to the intermediate holding spot -- where the message queue is located -- and a programming interface so you can put and get messages. This design also makes transfer between different platforms much easier. As long as a programming interface support program is available on the platform, messaging is possible. We ll see examples of that throughout this presentation. 4

5 WebSphere MQ Since 1992 WebSphere MQ (formerly MQSeries) has been a very widely used messaging infrastructure: Direct Connection (Bindings Mode) Remote Connection (Client Mode) Application C/C++, COBOL, PL/I, and Java MQ Code Queue Manager MQ Code Application C/C++, COBOL, PL/I, and Java Application JMS MQ Code Queue Queue Queue MQ Code Application JMS DB2 Assured Delivery End-to-End Security Simple API Wide Range of Platforms Queue Manager DB2 You can access MQ from WebSphere MQ can be JMS Provider (more in a bit) Question of Default Messaging and MQ is not either / or can be and Point-to-Point vs. Pub-Sub 5 IBM Americas Advanced Technical Support Let s just look briefly at WebSphere MQ Series, which has been around since 1992 or thereabouts and has been a very widely accepted messaging transport. MQ (as we ll call it) is based around the Queue Manager, which is middleware software that understands how to maintain message storage locations (called queues ) and knows how to back those queues to a persistent store (DB2) if needed. Multiple Queue Managers can be arranged into a kind of backbone of Queue Managers. They can also be clustered, and can exploit z/os data sharing facilities in a Sysplex environment. A program on one side forms up a message and passes it to the MQ API, which is what allows the message to make its way to the Queue Manager for holding. That program can be written in C/C++, COBOL, PL/I or Java, and the program can reside on a wide variety of platforms. We ll explore two ways programs get their messages to the Queue Manager -- when the program is on the same z/os LPAR as the Queue Manager, and when they re separated from the Queue Manager by a TCP network. The former allows cross-memory communications. There the MQ code invoked by the program is really just helper code to bind to the Queue Manager; hence the name Bindings Mode. That s a direct program-to-program connection. The latter involves putting the message request on a TCP connection and passing it to the Queue Manager. The program producing the messages still needs some MQ code -- that s what allows the communication with the Queue Manager, and that s what provides the API set. But because it s remote, it relies on the Queue Manager listening on a TCP port. This is called Client Mode. You can access MQ from WebSphere Application Server. Actually, there are several ways you can do this -- using the MQ APIs directly, or going through the open standard APIs, which is what JMS is all about. We have a lot to talk about JMS upcoming. Further, as you ll see, WebSphere MQ can be the JMS Provider to WebSphere, which means your existing MQ infrastructure can have a vital role even if you plan to use JMS. The point here is that the JMS stuff and MQ are not necessarily competing technologies. They can complement one another. 5

6 Point to Point vs. Publish/Subscribe (Pub-Sub) Two basic forms of messaging exchange: Point-to-Point Pub-Sub Publisher Publisher Program Program Topic Topic Queue Subscribers Primary program-to-program Programs generally know about each other Most common form of messaging Subscribers register interest in a topic Publishers post messages to the topics; subscribers then pull messages from interest topics Publishers generally don t know who subscribers are; they can register or de-register as they wish Both MQ and WebSphere Default Messaging provides this Application Put/Get vs. Message Driven Bean 6 IBM Americas Advanced Technical Support Here we ll explore two basic ways in which messaging can be handled. One is called point to point, and the other is called publish / subscribe. Both MQ and the Default Messaging of WebSphere Application Server support these. Point-to-point mesaging is where a program (the producer ) creates a message and places it on the queue. On the other side is another program, who knows the name of that queue and pulls the messages from there. If a message needs to go back, the flow is reversed. Often the queue is just for those two programs use. It s a point-to-point mechanism for message exchange between programs. (Sometimes there ll be two queues -- one for messages going left to right; another for messages going right-to-left in the picture.) Publish-Subscribe -- or pub/sub -- is a mechanism by which message producers put a message on a queue without any real knowledge of who ll consume that message. Subscribers register their interest in a queue -- called a topic in this setting -- and receive those messages from the topic space when the producer puts it there. The difference between the two is really based around the relationship between the producer of the message and the consumer of that message. In a point-to-point model that relationship is fairly tight; in a pub/sub model the relationship is fairly decoupled. 6

7 Applications and Retrieving Messages The more traditional method is a simple GET operation. An alternative is to for application to pull message when alerted something is in the queue Application logic decides when to attempt retrieval Program Program Retrieving application explicitely performs a get operation against the queue to retrieve the next message. Queue There may be nothing in the queue application then simply tries later. Application retrieves based on indication of message receipt Program Queue I have message! Program Application waits for signal that a message is arrived. Then it pulls the message and proceeds. In the JMS world, this is called a Message Driven Bean -- one that wakes up upon receipt of a message. Interface API vs. Transport layer 7 IBM Americas Advanced Technical Support There are two basic ways an application can receive a message from a queue: The first involves the receiving application -- the consumer -- programmatically connecting to the queue and seeing if anything is there. If so, it pulls the message; if not, it tries again later. The second involves the consumer being alerted to the fact a message has been received, and it waking up and pulling the message. In the JMS setting this is known as a Message Driven Bean -- its behavior is driven by the receipt of a message. Which of the two used is really an application design issue. 7

8 Java Message Service JMS 8 IBM Americas Advanced Technical Support 8

9 Messaging API vs. Messaging Transport This is an important distinction because it sets the stage for talking about JMS and how WebSphere implements messaging behind the JMS interface: Application Program Messaging System API Application Program Messaging System API As long as applications undertand messaging API, it doesn t need to know what s below the API. Boundary Message Transport Infrastructure Part that manages the holding and forwarding of messages Which means different transport options can be configured here And what we ll see is that WebSphere Application Server allows this -- you can use the default messaging (Java based) or you can use WebSphere MQ This is possible because of the separation between API and Transport JMS 9 IBM Americas Advanced Technical Support As we launch into the topic of JMS - Java Messaing Service -- it s important to get a critical concept on the table; that is, the difference between the messaging API and the messaging transport. The API is what the application program talks to when it wishes to pass or get a message. The transport is what actually stores the message and forwards it when requested. Understanding this distinction is critical because it helps understand how the lower-level transport can be provided separately from the API. That s exactly how WebSphere Application Server is designed to do this. WebSphere provides its own transport as part of the Application Server, but it s also possible to configure other transports. The most notable other transport is WebSphere MQ. The design is modular, with a boundary between the API and the transport. Any supported transport can be placed below the API and the applications above the API don t know and don t care. 9

10 JMS: An Interface Specification with Concepts Similar to MQ The JMS standard defines the standardized APIs (the interface ) for a Java EE environment. It employs concepts and terminology similar to WebSphere MQ: WebSphere Java Application JMS Interface (Standard APIs) Java Classes Java Classes Transport As implemented by the vendor Destination (Queue) or Topic (Pub/Sub) WebSphere Java Application JMS Interface (Standard APIs) Java Classes Java Classes Still have the notion of queues where messages are sent and retrieved Both point-to-point and pub/sub supported with JMS How the queues are actually implemented is left up to the vendor What implements the transport is known as the JMS Provider WebSphere Application Server provides four options JMS Providers 10 IBM Americas Advanced Technical Support Let s now look at Java Messaging Service, or JMS. This is really just an application programming interface for messaging with concepts similar to that offered by MQ or other messaging middleware -- concepts like queues (though they re called destinations ) and pub/sub (called topics ). The key thing is this -- JMS defines the interface; not the underlying transport. WebSphere Application Server supplies the interface as part of its offering of standardized services. The interface consists of a set of Java APIs and the supporting Java classes that implement the APIs. The supplied Java classes also provide the linkage to the underlying transport, whatever that may be. And that becomes the key focus right now -- what transport options does WebSphere Application Server support? There are four options 10

11 WebSphere Application Server and JMS Providers A JMS Provider is whatever software a vendor chooses to implement the messaging transport below (or behind ) the standard interface: WebSphere Java Environment WebSphere Java Application JMS Interface (Standard APIs) Java Classes Java Classes Message Transport Developer writes application to the JMS standard Java EE (WebSphere) implements the JMS interface Vendor provides options for transport implementation: WebSphere Application Server V6.1 Provides: Default messaging WebSphere Application Server s full-java messaging transport supplied with the product WebSphere MQ Configuring WebSphere Application Server to use WebSphere MQ as the messaging transport behind the JMS interface Generic Any third-party messaging system V5 default messaging Allows V6.x application servers to continue to use the V5 messaging provider (reduced function MQ) located with a WAS V5 application server WebSphere MQ as JMS Provider 11 IBM Americas Advanced Technical Support The JMS Provider is the software that implements the messaging transport behind the API. There are four basic options for the JMS provider: Default Messaging provider -- this is what comes with WebSphere. It is an all-java implementation of a messaging transport. We re going to cover this in some depth in this presentation. WebSphere MQ -- this can be configured to be the JMS provider. In essence it involves telling WebSphere how to reach the MQ Queue Manager. WebSphere simply relies on MQ to be the place where queues are maintained. Generic -- some other third-party messaging system. Configured in a way similar to MQ; that is, you need to tell WAS how to reach the messaging queue maintainer. V5 Default Message -- this is legacy code meant to provide a way for applications written to rely on the old V5 default messaging -- based on a reduced-function subset of MQ, provided with WAS V5. We ll focus on the first two. 11

12 WebSphere MQ as the JMS Provider Because of the separation between interface and transport, WebSphere allows MQ to be configured into the runtime to be the JMS Provider WebSphere Java Application WebSphere Environment WebSphere Java Application WebSphere comes packaged with JAR files that will allow client-mode connection without requiring access to MQ installables JMS Interface (Standard APIs) JMS Interface (Standard APIs) Or you can configure WebSphere AppServer to access MQ installation files: Java Classes Java Classes Java Classes Java Classes Java JAR files for client mode Java and native for bindings mode DB2 Queue Manager Queue Queue Queue The point here is that real MQ Queue Managers can serve as the JMS Provider for WebSphere Application Server Piggyback on existing messaging infrastructure taking advantage of qualities of WebSphere MQ The Default Messaging Provider 12 IBM Americas Advanced Technical Support WebSphere MQ can act as the JMS provider under the JMS interface because of the separation offered by the architecture. WebSphere provides a configuration selection that allows you to define a real MQ Queue Manager as the underlying provider. There are two ways to accomplish the connection to the MQ Queue Manager: You can use the Java class files packaged with WebSphere Application Server. This means you can connect to the Queue Manager without needing any information about the Queue Manager other than the host and port. You can configure WebSphere to access the MQ installation files to use those to access MQ in either client mode or bindings mode. The key point here is that MQ can serve as the underlying transport by configuring in WebSphere the ability to connect to the MQ Queue Manager -- client mode or bindings mode -- and using MQ as the transport. This means you can use your existing MQ infrastructure as the JMS provider. But the purpose of this presentation is to cover the Default Messaging Provider included with WebSphere. 12

13 Default Message Provider And the introduction of the SIB 13 IBM Americas Advanced Technical Support 13

14 WebSphere Application Server Default JMS Provider WebSphere Application Server comes with a messaging transport by default it is a pure-java implementation WebSphere Java Environment WebSphere Java Application JMS Interface (Standard APIs) Java Classes Java Classes Default Messaging Provider Queues/Topics Message Transport Security Persistence Don t have to install anything extra but you do have to configure it Involves the creation of a Service Integration Bus (SIBus), destinations (queues, topics), Queue Connection Factories and (if MDB) Activation Specification On z/os the creation of a SIBus is what triggers the starting of the Control Region Adjunct that s because that s where the Messaging Engine is implemented AppServer SR Typical AppServer Let s now look at the SIBus, Messaging Engines and other things related to the use of the Default Messaging Provider ADJ SR With SIB Messaging Engine Defined SIBus 14 IBM Americas Advanced Technical Support The Default JMS Provider is a set of functionality that is included with WebSphere Application Server. It s not configured at first; that s something you have to do, and that s done through the Administrative Console. When you configure the Default JMS Provider, you start by creating something called a Service Integration Bus (SIBus). Then you create destinations (queues for point-to-point; or topics for pubsub), Queue Connection Factories (which define how to reach the messaging provider), and if the application makes use of an Message Driven Bean, an activation specification, which tells the MDB which queue to listen on. Creation of the SIBus involves creating something called a messaging engine, which we ll discuss in a moment. When the SIBus and its messaging engine is configured, WebSphere z/os will know to start the Controller Adjunct Region, which is the address space that hosts the messaging engine. If you ve ever wondered what the Control Region Adjunct was for, this is it -- it s related to the SIBus and the Default Messaging Provider. Specifically, it hosts the messaging engine, which is the software implementation of part of the SIBus. But we re ahead of ourselves. What s an SIBus? 14

15 The Service Integration Bus -- a Logical Communication Structure This provides a framework for the hosting and exchange of messages within a WebSphere Application Server cell. Note: A single bus must stay within a cell. But a bus can be linked to another bus in another cell. LPAR A Daemon Node Agent DMGR SR AppServer ADJ SR LPAR B Daemon Node Agent Standalone servers may have configured buses as well. Showing ND here for completeness AppServer ADJ SR This is a logical structure the communications are really over the normal IP network that s used for WAS in general Not limited to just JMS -- is open to other messaging interfaces and models Bus itself is very easy to configure Cell may have multiple SIBuses Once the bus is defined, then you add bus members Servers Clusters MQ queue managers or queue sharing groups Can be a slippery concept because this is logical. Key is this is a way to define an administrative collection of resources associated with default messaging. Multiple SIBuses allows you to administratively isolate messaging resources if you wish Messaging Engines 15 IBM Americas Advanced Technical Support The Service Integration Bus (or SIBus) is a logical construction inside a WebSphere Application Server cell that provides a basic framework for the storing and forwarding of messages using just the Java infrastructure of WebSphere. We say it is logical because the SIBus itself is really just a definition -- it s not itself anything real. But things are then associated with the SIBus, and those things are real -- messaging engines and queue points. WebSphere needs the basics first, and that s the SIBus. The SIBus is not a separate communications network it is, as we said, just a logical thing mapped on top of the pre-existing TCP network that WebSphere relies upon. So when a message passes along the SIBus, it s really just passing along the TCP network underlying WebSphere. But from an administrative point of view, it s using the SIBus. We ll see at the end of the presentation that the SIBus itself is really pretty easy to configure. The other things -- members, messaging engines, destinations, foreign bus definitions -- they re a bit more complicated. But the bus itself is really easy. The bus definition comes first the other things are defintitions added to the bus once you define it. This notion of an SIBus can be a little difficult to grasp because it s logical. The key is to think of it just as a way WebSphere maintains the collection of messaging resources into a manageable grouping of things. That group is around the SIBus. You can have more than one SIBus in a cell. Or you can have just one. An SIBus can t jump to another cell, but it is possible to define a foreign link to an SIBus in another cell. We ll touch on that in a bit. 15

16 Messaging Engines Messaging engines provide the core messaging functionality. They reside physically in the z/os Control Region Adjunct of the Application Server Control Region Control Region Adjunct Messaging Engine 3 1 Servant Region Applications 1. Application access default messaging resources through the Messaging Engine Picture a bit misleading - it suggests applications must be in the same server as Messaging Engine not true. Applications can access any ME in the cell. 2. Messaging Engine is what provides connection to the SIBus, which really means it knows about other MEs (host and port) and knows how to talk to them 3. Messaging Engine is what maintains the message persistence store (if persistence desired): JDBC resource: DB2, Oracle, Sybase, Informix, SQL Server In V6.0 default is Cloudscape In V6.1 default is a non-jdbc file based persistence store Messaging Engine 2 Messaging Engine The Messaging Engine is automatically created when a server is named as a member of a SIBus Next time server started, the A will come up SIBus appilcations 16 IBM Americas Advanced Technical Support Finally we get to something that s real -- the messaging engine. This is function implemented in WebSphere software that provides the working pieces of the messaging infrastructure. The are created when you add a member to a bus. So for example, if you add server XYZ to a SIBus you just defined, WebSphere will know to modify the configuration so that server will then have a messaging engine. The engine itself will be hosted in a Control Region Adjunct, which is another address space like a servant region in that it s started automatically by WLM. Unlike the servant region, you can only have one. Messaging engines are the connection point to the bus for the purpose of placing or getting messages. An application needs to be able to reach and connect with a messaging engine so the ME can do the work of locating the queue and handling the messages. This picture is misleading in that it implies the application needs to be in a server that has a ME, but that s not really true. There are several different ways an application can access the SIBus, as we ll see in an upcoming chart. The main point is that an application needs something to serve as the real point of contact for the SIBus the SIBus itself is, as you may recall, a logical structure. But the messaging engine is real. And applications access the SIBus through a messaging engine. Messaging engines are also what maintain the physical message persistence store, be that a JDBC connection to a database or a file-based store. Once again we need to be careful -- you can connect to any ME on the SIBus to place/get a message somewhere on the bus. If the message store is not on the ME the application connects to, the SIBus (in reality, WebSphere Application Server) will locate the ME where the message queue is maintained and route the request there. The messaging engine is not something you have to create it s done automatically when a server (or cluster) is added as a member of an SIBus. If you ve ever wondered when the A would start up, this is it -- when a server is added as a member to an SIBus, the ME is defined. The next time that server is started, WAS sees the server has an ME and it fires up the A. 16

17 SIBus Applications There are a number of different ways in which a messaging application can connect to the SIBus: LPAR A Daemon Node Agent 2 AppServer SR DMGR SR AppServer ADJ SR 1 3 EJB or Servlet in another WAS cell SIBus Link 4 JMS Application in a J2SE JMS 1.1 container 5 JMS Application in a Client Container C/C++ and C# using XMS Bootstrap into the WAS environment and connect to the SIBus 6 1. EJB or servlet in an application server where the messaging engine is running 2. EJB or servlet in an application server elsewhere in the cell where the SIBus is defined 3. EJB or servlet in another WebSphere cell May have its own bus with a foreign destination defined. More on this in a bit. 4. Servlet in any standard J2SE JMS 1.1 compliant container IBM Client for JMS on J2SE with IBM WebSphere Application Server freely downloadable at: ibm.com/support/docview.wss?uid=swg JMS application in a client container, which is outside of an application server 6. Non-Java environment using IBM Message Service API (XMS) Freely downloadable from: ibm.com/support/docview.wss?uid=swg Message: many options, not just from applications in server where ME is running Destinations 17 IBM Americas Advanced Technical Support Let s now look at the several different ways an application can reach the SIBus and make use of the messaging infrastructure. 1.The first is the most obvious -- the application is located in the same application server where the messaging engine is located. The application makes it connection to the SIBus through a Queue Connection Factory, (QCF) which is a definition you define under the Resources section of the Administrative Console. We ll see that later in this presentation. The QCF is what provides the ability to connect to the SIBus. This is really little different from #2 2.Any server in the cell is capable of connecting to the SIBus through a defined Queue Connection Factory. This illustrates the earlier point that an application does not need to be deployed in the same application server as the messaging engine. 3.Applications in other WebSphere cells can connect to their own SIBuses and make use of messaging services in another cell, but not directly. They would require an SIBus link, which is a set of definitions you create in both cells that tell each SIBus about the other. That involves telling it host:port of the server that will serve as the gateway into the SIBus. That s a somewhat complicated topic to go into in any more detail than that right now. 4.JMS applications in a servlet container such as Tomcat that doesn t have direct access to the SIBus may use a freely downloadable bit of client code that will help with the bootstrapping into the SIBus to access messaging services. 5.Standalone clients boostrap into the environment to gain access to the SIBus as well. 6.Even non-java applications can access the SIBus using another dowloadable package. Do you see the message (pardon the pun) coming through here? Applications can reside in many different places and use the services of the SIBus, not just the server where a messaging engine is located. In a sense this is like WMQ, where applications can access the Queue Manager from a number of different locations. 17

18 SIBus Destinations Now we get to what most people think of when they think of messaging -- queues and pub/sub topic spaces: Application SIBus In MQ a queue is something the Queue Manager implements, typically in a backing DB2 table In Default Messaging, a destination is the same in concept, but the implementation is a bit more layered Destination Queue or Topic space A virtual location for handling messages An application points to a queue or topic space. The actual implementation is hidden behind the virtual location Message Point Software implementation of the destination This is hosted inside of a messaging engine. If the queue is in a cluster (more in a bit) then a destination may have multiple message points Persistence Implementation (If defined) Message point may be backed by persistence implementation Supported JDBC data base or in V6.1 a file based implementation (Or nothing -- you may opt to have messages non-persistent) Five types of destinations 18 IBM Americas Advanced Technical Support In the world of SIBus messaging, we have the notion of a destination, which can be thought of as roughly analogous to a WMQ queue, but there s a bit of abstraction involved. A destination is really a logical (or virtual ) location that is used for addressing purposes. An application is configured to make use of a destination, which may take the form of a queue (point to point) or a topic (pub/sub). That logical destination is then mapped to a physical message storage location known as a message point. The message point is maintained by the sofware inside the messaging engine. From an application point of view the physical location of the message point is not important. All the application cares about is connecting to the destination, and the SIBus then takes care to locate and route the request to the place where the actual message point is maintained. Below that is the persistence implementation if that s configured. (As we ll see in a bit, you don t have to persist messages you can have them in memory only if you like. That makes things faster, but at a cost of reliability. That s the age-old trade-off.) As noted earlier, if you store messages they can be in a file-based (which is new with V6.1, and is really easy to set up so it s probably the best way to go when you re first playing with the SIBus stuff), or a JDBC connection to a real database behind the scenes. This looks complicated, but what s really going on here is the rather standard abstraction layering so application resources don t need to be directly tied to physical implementations. 18

19 Five Types of SIBus Destinations From an application s perspective, all essentially the same. But from an administrative perspective they re all a bit different: Explicit A created and named destination that applications point to by that name. Alias Is a way to hide an actual destination behind an alias point. Changes to the destination can be made without affecting the application. Foreign A definition of a destination that s really located elsewhere -- another SIBus, or an MQ queue. This also involves the creation of a foreign bus definition and an SIBus link. More in a bit. Temporary Applications can use a defined API and create a destination on the fly for use during the life of that application. You don t see this option on the Administrative Console. Exception Where messages go that encounter exception condtions. This is similar to the Dead Letter Queue in MQ. This is automatically created by WebSphere. Message points 19 IBM Americas Advanced Technical Support There are five types of destinations all more or less the same thing, but they each have slightly different implementations. The chart provides most of the necessary commentary. When you create a destination you ll see these distinctions offered. You select which type you re looking to create and the wizard will lead you through the process from there. 19

20 Message Points - Where Queue or Topic Physically Maintained When a destination or topic is created, you specify where the queue or topic will be physically maintained Reminder of the steps involved Server 1 A Mess. Eng. Create the bus Simple definition of a logical structure. No physical implementation until you start assigning members to the bus. SR SR A Server 2 A Mess. Eng. B C SIBus Assign members to the bus Server of cluster (more on this next). This is what creates the messaging engine, and on z/os this is what results in the A starting up. Create destination and assign to a messaging engine This creates the physical implementation of the queue or topic, which is maintained by the assigned messaging engine Important -- the queue or topic is accessible by applications running elsewhere. This does not limit accessibility. Server 3 SR A Mess. Eng. D But it may limit availability and scalability. If the server where ME hosted goes down, that message point is lost unless clustering used SIBus and clustering 20 IBM Americas Advanced Technical Support As the earlier chart indicated, destinations have a physical implementation behind the logical definition. When you create a destination, you also pick where the message point (the physical implementation) will be hosted. What you are doing is picking the messaging engine where the message point will be maintained. There s two key points to make: The message point has to be maintained somewhere, and the design of the SIBus does not allow every message point to be created/maintained in all messaging engines all at once. That would be a nightmare it would imply replicating and copying messages to all the different messaging engines to maintain data concurrency. This does not mean there s no HA abilility; there is, as we ll see with clustering. The point here is that the physical message point (think WMQ queue) has to be hosted somewhere. A given ME is where that takes place. An application can access any ME along the SIBus, so there s no need to worry about application proximity to the actual place where the message point is maintained. Remember -- applications address themselves to the logical definition. WebSphere then maps that to the message point and locates the ME where it s physically maintained. We bring this up because it gets to the question of availability and scalability. With a single message point you face the possibility of data inaccessibility if that messaging engine goes does. But you can overcome that with clustering, which we ll see. The scalability issue can be overcome by partitioning data across multiple message points in a cluster, but that topic is well beyond the scope of this presentation. But clustering is not beyond the scope, so let s take a brief look at that. 20

21 SIBus and Clustering A WebSphere cluster typically consists of two or more servers. From SIBus perspective, it represents a single Bus Member. That brings up some interesting questions about where messaging engines created Failover Concurrent Server 1 Server 2 Server 1 Server 2 A Mess. Eng. A Mess. Eng. A Mess. Eng. A Mess. Eng. SR A SR SR A SR A HAM Failover Persistence Store Persistence Store Cluster added as Bus member, but messaging engine created in only one member of the cluster On z/os A started in all members to accept ME if original-hosting server lost WAS High Availability Manager (HAM) responsible for moving ME function (and message points) to next server in cluster All resources (destinations, message points) move. WLM will route to the new location if failover occurs Persistence store must be accessible from all servers for this to work Cluster added as Bus member, and configured so ME created in two or more of the cluster members Destinations assigned to a cluster with multiple MEs will result in queue being partitioned across the MEs How message producers, consumers, MDBs and pub/sub works in this environment is complex and beyond the scope of this presentation. Message: Availability and Scalability Foreign buses and destinations 21 IBM Americas Advanced Technical Support A WebSphere Application Server cluster is two or more servers arranged into a logical one for the purposes of administration and operations. Physically they are separate servers. Logically and administratively they are in many ways treated as one. From an SIBus perspective, we face two different ways in which the cluster can participate in the bus. Recall that when adding a member to an SIBus, the member can be an individual server or a cluster. If you add a cluster as a bus member, by default WebSphere will create the ME in only one of the servers. But, you say. That means it s not HA! Well, maybe, maybe not. Part of WebSphere s design is the ability to failover singletons -- single instances of functions. A messaging engine is an example of a singleton service. The WebSphere High Availability Manager (HAM) functionality is design to recognize when singleton services are lost and automatically start them on other components in the cell. In this example the HAM would failover the ME to the other cluster member. (Clusters, when added to an SIBus, result in the Adjunct Region started for both members so it stands ready and able to take over the ME if needed.) If a failover occurs, then WebSphere tracks the movement of the messaging engine, and all the associated message points, and will make sure that subsequent requests for those resources get routed to the new location for the ME. Two quick points: Have you ever heard people talk about turning off HAM? If you do, then this ability to failover the ME to the other cluster member is lost. If you ve defined a persistence store for the messages, that store needs to be accessible from the other cluster member in the event the failover occurs. An alternative is to let WebSphere create ME s in both cluster members. Then message points get partitioned, with some messages stored in one and other messages in others. That s a more advanced topic and beyond the scope of this presentation. 21

22 Foreign Buses and Foreign Destinations One SIBus may talk to another -- in the same cell or in other cells on other servers. Process involves defining the foreign resources on both ends: Actual step-by-step process to configure beyond scope of what we can show here Server Server Server Server ME ME ME ME Create Foreign Bus Definition Name of the foreign link Local Messaging Engine to serve as gateway to other side Name of the SIB on other other side Messaging Engine that serves as GW on the other side Host, port and transport chain details Create Foreign Destination or Topic Assign to the foreign bus definition Create very similar definition on the other side, but pointing back to the first SIBus Interesting point - message producers can put a message on a foreign queue; message consumers can only consume from a local queue. Two applications over a foreign link would have to produce to each other to the foreign side. Then consumption can take place. Two buses in the same cell very similar, but Administrative Console helps because it can see the other bus. SIBus in another cell requires you to provide values by hand. You can also define MQ as a foreign bus You may route messages through a SIBus or WMQ on way to final bus destination Resources 22 IBM Americas Advanced Technical Support We mentioned earlier that an SIBus could not span multiple cells. That s because WebSphere needs to see the environment to operate the SIBus, and if a bus were to jump multiple cells it would be difficult to achieve that. But that doesn t mean applications can t make use of messaging resources in other cells. They can. It s just that they have to make use of something called a foreign bus definition to bridge over to the other SIBus. Note: two separate SIBuses in the same cell have the same issues. You would still treat one as foreign to the other. But the creation of the foreign bus links is made much easier when the two buses are in the same cell because the Administrative appliction can see both bus definitions and help you connect the two. Configuring the foreign bus links is really something done on both sides of the connection. The chart illustrates (at a high level) the process from one side, and you d do the same from the other. This is very much like the process you d do in other systems where you re defining a channel or link between two systems -- both sides need to know about each other. If you have the real WebSphere MQ somewhere in your environment, you can define it as a foreign bus to a SIBus. From the MQ side the SIBus looks like another Queue Manager. You can also use a foreign bus (or MQ for that matter) to act as a hub for messages passing through to another bus. From an application point of view, destinations on other buses are addressed with a foreign destination (we saw that as one of the options a few moments ago). The foreign destination is defined with a pointer ot the foreign bus link that s what tells the SIBus to route the request over the foreign bus link to the other SIBus (or MQ), where the request is mapped to the actual destination or queue there. Yes, this can get complicated. But the capabilities are there. Note the small-print comment about producers and consumers of messages. Producers can push a message over to a foreign bus, but a consumer can t reach across and pull one. You d need applications on both sides producing and sending to the other side. Consumers there could then get from their local destination. 22

23 Resources - QCFs, Queues and Activation Specs After the SIBus and its definitions are created, the Resources are created which link to the bus. It s the resources the applications bind to. MDB application and Listener Port / Act Spec binding Queue Connection Factory Defines how to get to the JMS provider, be that the MQ Queue Manager, or the SIBus. Carries a JNDI name applications bind to. Queues or Topics Defines the MQ queue or topic space on the SIBus destination to use. Carries a JNDI name the application binds to. Activation Specification Used by Message Driven Beans when the SIBus is where the queue resides. This definition ties together the QCF (SIBus pointer) and the queue definition and carries a JNDI name the MDB binds to. For MDBs where MQ is the JMS provider, a Listener Port (part of the server definition) is used to tie the QCF and the queue into a single listener point Resource references to queue definitions QoS 23 IBM Americas Advanced Technical Support Let s move up a layer and take a look at how applications make use of the SIBus reources we just got done talking about. Here it s important to stop and reflect what we have is a lower level messaging infrastructure (SIBus, messaging engines, destinations, message points, etc.), and the higher-level JMS layer (which is going to consist of queue connection factories, queue or topic definitions and activation specifications). Key Point: The application does not see the underlying transport infrastructure. It sees the JMS interface. The lower transport is accessed through the definitions at the JMS level. That s key because the lower transport does not need to be the SIBus it can be MQ or even a third-party message provider. Abstraction separation that s the key. The chart provides a quick summary of some of the definitions up at the JMS level, and some illustrations of what happens when you deploy a JMS application. Queue Connection Factory -- this provides a definition of how to reach the messaging transport. This definition will either specify the SIBus or the WMQ queue manager. This is assigned a JNDI name, and when you deploy an application you tell it the JNDI name of the queue connection factory so it can look it up and connect to the messaging resources. Queue -- this provides a definition of the specific queue (or topic) resource. This too carries a JNDI name and the application binds to this. This definition maps to the actual WMQ queue name, or the SIBus queue or topic. So the application is bound to the JNDI name, which then maps the request to the actual queue in the transport, using the queue connection factory to know how to reach the transport. Activation specifications -- this is a definition that provides information about what queue to listen on for Message Driven Beans. The Activation Spec has a pointer to the QCF and the queue, and the application is mapped to the JNDI name of the Activation Spec. The application MDB then listens on that connection and reacts when a message is received. 23

24 Messaging Quality of Service Five levels of QoS all related to the nature of message persistence. This is defined on each destination you create: Best Effort Non-Persistent Never written to disk. Silently thrown away if memory cache overruns Reliability Performance Express Non-Persistent Reliable Non-Persistent Reliable Persistent Assured Persistent Occasionally written asynchronously to storage when memory cache overruns. May be lost due to resource failures or shortages. Does not persist across restarts Occasionally written asynchronously to storage when memory cache overruns. Does not persist across restarts Written asynchronously to persistent storage. Messages are recovered on restart Written synchronously to persistent storage in line with transactional boundaries. Messages are recovered on restart Intended as a reference chart we won t go through each of these Message mediation 24 IBM Americas Advanced Technical Support We briefly mentioned before how there are differing levels of persistence for message, and that helps determine the quality of service you can get. We are not going to go through all these other than to offer this as a reference. The key message here is really just that the more robust the persistence the less speed that can be achieved, but the more reliability you have. The reverse is true in the opposite direction. Which you choose is based on the needs of the application and the QoS you re looking to provide. 24

25 Message Mediation WebSphere SIBus provides a kind of user exit structure for your custom code to intercept and modify messages before they reach the consumer Optional! Don t have to use it function/capability there if you need it. Message Producer Message Consumer Mediation Chain Mediation Handler Mediation Handler Mediation Point Queue Point Destination We mention this for two reasons: 1. It illustrates the flexibility of the model, allowing you to transform messages between producer and consumer 2. To point out that another IBM product takes full advantage of this -- WebSphere Enterprise Service Bus (WESB) Custom Java beans you write to do message modification. None, one or many WESB comes with a package of pre-built mediation primitives 25 IBM Americas Advanced Technical Support Interoperating with MQ What we re going to talk about here is something interesting about the SIBus structure, but not something you need to worry about when you re first starting out with default messaging. The architecture of the default message provider allows for messages to be intercepted and modified if you wish to do it. Key point -- you don t have to do this it s just there if you want to. You can think of this like a user exit it is a defined point where you can hook in your own custom code to intercept the message and modify it before it goes to the queue. The architecture defines a mediation chain which is some number of your written Java beans that get invoked in order to modify the message. We bring this up because other IBM solutions make use of this -- most notably WebSphere Enterprise Service Bus (WESB), which uses this function for its message mediation (fancy word for changing a message on the fly). The WESB product takes this a step further by providing a set of primitives (pre-written mediation program building blocks) that can be used to modify messages on the fly. 25

26 Interoperating with WebSphere MQ 26 IBM Americas Advanced Technical Support 26

27 Multiple Ways of Using WebSphere MQ with SIBus Three basic ways this can be done: 1. Use WMQ as the JMS provider We saw this earlier. WMQ can be configured as the underlying JMS transport. Doing this means no SIBuses and no Adjunct regions WMQ acts in place of SIBus. 2. Configure a foreign bus and point to WMQ queue manager SIBus links over to WMQ and vice-versa. SIBus sees WMQ as another SIBus; WMQ sees SIBus as another QMGR. 3. In V6.1 and V7, configure a WMQ Server and add it as a new bus member: Server ME Server ME WebSphere MQ QMGR Queues and Topic Spaces Define WMQ server under Servers in Administrative Console Provide information about QMGR, host, port and channel information Add new WMQ server as a bus member It will show up in the dropdown list of eligible servers for inclusion as a bus member Create a destination and assign it to the new WMQ bus member Part of creating a destination is the assignment to a messaging engine. Administrative Console will now show WMQ server as an eligible ME for assignment of the queue. The message point is not created on the SIBus it is assumed to actually exist over in the WMQ QMGR. Administation and Creating the SIBus 27 IBM Americas Advanced Technical Support We ve mentioned WebSphere MQ several times during this presentation, so this chart will serve to remind us of the different ways in which WMQ can be used as part of this JMS question. 1.WebSphere MQ can be used as the JMS provider. By this we mean you can bypass the use of the SIBus altogether and just configure a WMQ queue manager as the provider. Your Queue Connection Factories (used by applications to know how to reach the JMS transport) will have information about the WMQ queue manager, either bindings mode information or client mode information. WebSphere simply passes the requests along to WMQ and the application is satisfied. 2.WebSphere MQ can be defined as a foreign bus to a SIBus. A foreign bus link defines to WebSphere how to reach and communicate with the foreign bus. In the case of WMQ as the foreign bus, the SIBus thinks it s talking to another SIBus, and in reverse WMQ thinks it s talking to another Queue Manager. 3.The final one is using WMQ as a kind of messaging engine on the SIBus. You can create a definition within WebSphere that names an MQ queue manager and provides connection properties to it. Then you can add a member to the SIBus and select from the pulldown menu the WMQ server you just defined. To the SIBus it s just another bus member and messaging engine, but in reality it s WMQ. 27