Infrastructure Series TechDc WebSphere Message Brker / IBM Integratin Bus Parallel Prcessing (Aggregatin) (Message Flw Develpment) February 2015 Authr(s): - IBM Message Brker - Develpment Parallel Prcessing (Aggregatin) Glen Brumbaugh
Table f Cntents Intrductin... 3 Dcument Versin... 3 Prduct Naming Histry... 3 Prduct Cmpnent Terminlgy... 3 Parallel Prcessing... 4 Prgramming Overview... 4 Message Brker Aggregatin Implementatin... 5 Queues... 5 Fan Out Message Flw... 6 Fan In Message Flw... 7 Errr Handling... 8 Slutin Design Summary... 9 Message Brker Aggregatin Ndes... 10 AggregateCntrl Nde... 10 AggregateRequest Nde... 11 AggregateReply Nde... 11 Best Practices... 11 References... 12 Table f Figures Figure 1 - Parallel Prgramming... 4 Figure 2 - Aggreatin message prcessing... 5 Figure 3 - WMQ Aggregatin Prcessing... 6 Figure 4 AggregateCntrl Nde OuputLcalEnvirnment... 7 Figure 5 MQOutput Nde OutputLcalEnvirnment... 8 Figure 6 AggregateRequest Nde - OutputEnvrinment... 9 Figure 7 - Aggregatin Message Tree... 10 Page 2 f 13
Intrductin Dcument Versin This dcument describes hw t utilize parallel prcessing (Aggregatin) within a Message Brker (WMB up t v8.x) r Integratin Nde (IIB v9.0 +) Message Flw. The dcument shuld, hwever, apply t mst versins f these prducts. The cntents f this dcument have been specifically verified n the fllwing prductin versins: WebSphere Message Brker v7.0.0.2 IBM Integratin Bus v9.0.0.0 This dcumentatin has been created and maintained by: Glen Brumbaugh This dcumentatin has been reviewed by: Glen Brumbaugh This dcumentatin was last updated: Versin 1.0 February 2015 Prduct Naming Histry The prduct currently knwn as IBM Integratin Bus has been thrugh a number f different prduct names during its several decade lng evlutin. The prduct was riginally develped by the New Era f Netwrks (NEON) Crpratin and was marketed and resld by IBM. IBM cmpletely redesigned and rebuilt the prduct and released their wn in-huse develped prduct beginning with versin 2.0. The prduct has had the fllwing names and versin numbers: MQSeries Integratr (MQSI) Versin 1.0 2.0 WebSphere MQSeries Integratr Versin 2.1 WebSphere Business Integratin Message Brker (WBIMB) V5.0 WebSphere Message Brker (WMB) Versin 6.0-8.x IBM Integratin Bus (IIB) Versin 9.0-10.0 Fr the remainder f this dcument, the prduct will be referred t as Message Brker. This is bth fr histrical reasns and t signify that this dcumentatin applies t bth the WMB and IIB prduct versins. Prduct Cmpnent Terminlgy With the Versin 9.0 prduct rename (t IBM Integratin Bus), several key prduct architectural cmpnents were given new names; while cntinuing t fill virtually the same rle they had previusly filled. This dcumentatin will cntinue t refer t the ld names because the steps dcumented here refer t bth ld and new prduct versins. The ld and crrespnding new names are as fllws: Message Brker Nw called Integratin Nde (Beginning with v9.0) Executin Grup Nw called Integratin Server (Beginning with v9.0) Message Flw Still called Message Flw Page 3 f 13
Parallel Prcessing Prgramming Overview Mst applicatin prgramming, whether in the C, COBOL, Java, r Message Brker prgramming languages, deals with sequential prcessing within a single prcess/thread. Very few prgrammers actually have experience with parallel prcessing, either within a multi-threaded envirnment r within a prgramming language that explicitly supprts parallel prcessing. Parallel, r cncurrent, prgramming has, hwever, been arund fr quite sme time and has been implemented in a number f different prgramming languages. Parallel prcessing (as illustrated in Figure 1) invlves tw separate peratins. The Frk peratin splits the main prcess thread int multiple parallel sub-threads. These child threads begin executin immediately. Once all child threads have been launched the parent thread suspends its prcessing until all the child threads have successfully cmpleted. Each child thread will terminate with a Jin peratin. Once all child threads have terminated with a Jin, the parent thread will resume prcessing. Figure 1 - Parallel Prgramming Naturally, unexpected errrs can ccur during the whle f this prcess. The primary errr assciated with parallel prcessing is the case f a child thread that never cmpletes with a Jin peratin, leaving the Frk waiting. This may be due t either a design/prgramming errr r the death f the child thread. This errr case is usually handled by resuming the parent thread after a certain amunt has passed. This is typically accmplished by using a timeut parameter. T summarize: Parallel prgramming uses the Frk and Jin prgramming cnstructs. Parent threads issue the Frk peratin, create child threads, and are then suspended. Page 4 f 13
Child threads created by the Frk peratin terminate with a Jin peratin. The parent thread resumes either when all child thread cmplete r a timeut is reached. The failure cnditin f nt all child threads successfully cmpleting must be handled. Message Brker Aggregatin Implementatin Since Message Brker is a messaging based technlgy, it uses messaging terminlgy t describe its parallel prcessing. T begin with, the term Parallel prcessing is nt used. Instead, Message Brker defines the cncept f Aggregatin. What in parallel prgramming languages is called a Frk, Message Brker terms a Fan Out. What parallel prgramming language calls a Jin, Message Brker terms a Fan In. The data trees frm the Fan Out messages are aggregated during the Fan In prcess. Message Brker supprts parallel prcessing thrugh a defined set f ndes built int the Message Brker tlkit. These ndes are nrmally implemented acrss tw separate Message Flws. The first flw is the Fan Out flw. The secnd Message Flw is the Fan In flw. In between these tw flws, the Fanned Out messages will perfrm the necessary business prcessing and generate Reply messages that will be aggregated in the Fan In flw. Message Brker uses three different ndes t implement Aggregatin. The tw separate flws and the assciated three ndes are referenced in Figure 2 and Figure 3. The three Message Brker Tlkit ndes are: AggregateCntrl Nde AggregateRequest Nde AggregateReply Nde Figure 2 - Aggregatin message prcessing Queues Message Brker aggregatin internally uses WMQ queues t supprt aggregatin prcessing. The queues used are: SYSTEM.BROKER.AGGR.CONTROL SYSTEM.BROKER.AGGR.REPLY SYSTEM.BROKER.AGGR.REQUEST SYSTEM.BROKER.AGGR.TIMEOUT SYSTEM.BROKER.AGGR.UNKNOWN Page 5 f 13
Fr islatin and perfrmance, it is pssible t cnfiguratin an Aggregatin t use alternate queues. This is dne thrugh an Aggregatin Cnfigurable Service. In this case, the Cnfigurable Service creates a prefix that is used t identify differently named queues. The resulting queue names are: SYSTEM.BROKER.AGGR.prefix.xxx (Where xxx = CONTROL, REPLY, etc.) Fr mre infrmatin, the References Sectin cntains a Knwledge Center link t this tpic. Figure 3 - WMQ Aggregatin Prcessing Fan Out Message Flw The Fan Out Message Flw begins with a request message frm a user. This request is prcessed by an Input nde. This input nde begins a transactin (Lgical Unit f Wrk) that will nt be cmmitted until the AggregateRequest ndes. The actual Fan Out (Frk) prcess itself begins in the AggregateCntrl nde. This nde allws multiple ndes t be cnnected t its Out terminal. These multiple Out cnnectins define the scpe f the Fan Out (parallel prcessing). The nde als defines the Aggregatin Name as a Nde prperty, s that data frm different aggregatins can be kept separated in the queues. At runtime, this Nde defines the Aggregatin ID, which enables the separatin f data Page 6 f 13
frm multiple instances f the same aggregatin. Bth f these fields are stred in the OutputLcalEnvirnment tree (See Figure 4). The parallel Fan Out ndes must each generate a Request message that will be prcessed utside f the Fan Out message flw. Each f these Request messages will, in turn, generate a Reply message that will be input int the Fan In Message Flw. When these Request messages are generated, typically by an MQOutput nde, the nde updates the OutputLcalEnvirnment element, adding a WrittenDestinatin element. This is illustrated in Figure 5. Figure 4 AggregateCntrl Nde OuputLcalEnvirnment Each Fan Out (Child) prcess ends with an AggregateRequest nde. The AggregateRequest nde prvides a name (called a Flder ) fr each f instance f a child prcess. These Flder names will be aggregated under a CmIbmAggregateReplyBdy dmain during the Fan In prcess. This nde als checks t see if it is the last Fan Out message flw. If it is, this nde cmmits the transactin. This cmmits cnsumes the riginal request message and releases the request messages frm each f the Fan Out flws. This nde als updates the OutputEnvirnment element. This structure is illustrated in Figure 6. The Envirnment element fields updated include: Aggregatin Name Flder Name Reply Identifier Reply Prtcl (Element name; frm AggregateCntrl nde prperty) (Element name; frm AggregateRequest nde prperty) (Matches value set in Output nde) (Typically MQ ) Fan In Message Flw The actual Fan In prcess itself begins with an Input nde t prcess the Reply messages generated in respnse t the Fan Out request messages. The Input nde begins a transactin (Lgical Unit f Wrk) that will nt be cmmitted until the final Output nde. The Fan In (Jin) prcess itself begins and ends with the AggregateReply nde. This nde cnsumes the Reply messages that are generated as a result f the multiple Request messages generated during the Fan Out prcessing. The nde aggregates all f these Reply messages int the CmIbmAggregateReplyBdy element. Under that element, each Reply is stred under a named Flder. The name f that flder is the name assigned by the Page 7 f 13
crrespnding AggregateRequest nde. illustrated in Figure 7. The CmIbmAggregateReplyBdy structure is Figure 5 MQOutput Nde OutputLcalEnvirnment Errr Handling There are a number f different types f errrs that may ccur in an aggregatin Message Flw. These errrs include: A Reply message that cannt be assciated with a request ( Unknwn ). A late (if there is a specified timeut) Reply message ( Timeut ). A drpped Reply message, causing either a timeut r the aggregatin t hang. A dwnstream errr (after AggregatinReply nde) causing an exceptin t be thrwn. All f the errrs must be handled in the fllwing Fan In Message Flw nde terminals: Input Nde.Catch terminal AggregateReply.Catch terminal (e.g. MQInput nde) Page 8 f 13
If these errrs are nt caught in thse tw ndes, then default Message Brker errr handling will be perfrmed. In the References Sectin, see the link regarding Errr Handling fr additinal infrmatin. Figure 6 AggregateRequest Nde - OutputEnvrinment Slutin Design Summary The design f an Aggregatin slutin thus has the fllwing prperties: The slutin has tw separate Message Flws: Fan Out and Fan In. Each Fan Out Message Flw: Begins with a single Input nde. The Fan Out prcess begins with an AggregateCntrl nde. Each Fan Out parallel flw has an Output nde generating a Request message. Each Fan Out parallel flw Output nde is fllwed by an AggregateRequest nde. The Request messages generated in the Fan Out parallel flws: Must ultimately result in a Reply message being generated. All Reply messages must be ruted t the same queue. The Reply queue is used as the Input queue fr the Fan In Message Flw. Page 9 f 13
Each Fan In Message Flw: Begins with a single Input nde. All Reply messages must ultimately be ruted t this Input nde. The AggregateReply nde perfrms the Fan In functin. Reply messages aggregated int a CmIbmAggregateReplyBdy structure. Figure 7 - Aggregatin Message Tree Message Brker Aggregatin Ndes AggregateCntrl Nde The AggregateCntrl nde is used in the Fan Out message flw. This nde must be preceded by an Input nde. This nde must precede the AggregateRequest nde. The input nde defines the beginning f a transactin. The AggregateCntrl nde must participate in the transactin. This nde begins the Fan Out prcess. Page 10 f 13
Fan Out parallel flws are defined by multiple utputs frm the Out terminal. This nde names this particular Aggregatin. Updates the OutputLcalEnvirnment tree. Adds a CmIbmAggregateCntrlNde child element. Adds CmIbmAggregateCntrlNde child fields. aggregatename aggregateid All parallel flws are passed identical tree structures. AggregateRequest Nde The AggregateRequest nde is used in the Fan Out message flw. This nde must be preceded by the AggregateCntrl nde. This nde must be preceded by an Output nde. All Output messages must ultimately result in a Reply message. All Reply messages must ultimately be ruted t the same queue. The AggregateRequest nde defines the end f a transactin. Transactin cmmitted when the last parallel AggregateRequest nde is prcessed. This nde names the Flder assciated with this parallel Request message flw. AggregateReply Nde The AggregateRequest nde is used in the Fan In message flw. This nde must be preceded by an Input nde. Reply messages resulting frm Fan Out parallel flws rute t this Input nde. The AggregateRequest nde builds the CmIbmAggregateReplyBdy structure. The CmIbmAggregateReplyBdy has a Flder element fr each Reply message. The name f the Flder element is assigned in the AggregateRequest nde. The Flder element crrespnds t the OutputRt f the Reply messages. The UnknwnMessageTimeut prperty Delays prcessing a Reply message fr which there is n cntrl infrmatin Best Practices MQInput Nde ( Fan Out ): Transactin mde MUST be set t Yes. Subsequent ndes MUST use a setting f Autmatic. This ensures that the input messages are prcessed under a syncpint and nly cmmitted when the last message is prcessed by the AggregateRequest nde. This accmplish tw essential things: (1) Data integrity is ensured. Input Request messages are rlled back nt the input queue if the transactin is rlled back. (2) N Request messages are released until the end f the transactin. This ensures that the cntrl infrmatin required by the AggregateReply nde is supplied befre any f the Reply messages arrive. AggregateCntrl Nde: D nt use the Cntrl terminal. This terminal is deprecated and remains nly t supprt earlier prduct versins. The use f this terminal has n benefit and Page 11 f 13
can cause perfrmance degradatin. In the References Sectin, see the link regarding Thread Starvatin. MQInput Nde ( Fan In ): Transactin mde MUST be set t Yes. Subsequent ndes MUST use a setting f Autmatic. This ensures that the input messages are prcessed under a syncpint and nly cmmitted after the final Reply message is written, fr example, by an MQOutput nde. This accmplish tw essential things: (1) Data integrity is ensured. Input Reply messages are rlled back nt the input queue if the transactin is rlled back. (2) Since exceptins may be thrwn t either the Input nde r the AggregateReply nde, this ensures a cnsistent behavir acrss different types f exceptins. References IBM WebSphere Message Brker Knwledge center (v7.0, 8.0) 01.ibm.cm/supprt/knwledgecenter/#!/SSKM8N/mapfiles/prduct_welcme.html IBM IBM Integratin Bus Knwledge center (v9.0, 10.0) 01.ibm.cm/supprt/knwledgecenter/#!/SSMKHH/mapfiles/prduct_welcme.html IBM IIB Knwledge center Aggregatin Assciating Fan Out and Fan In Message Flws 01.ibm.cm/supprt/knwledgecenter/#!/SSKM8N_8.0.0/cm.ibm.etls.mft.dc/ac12311_.ht m IBM IIB Knwledge center Aggregatin Creating the aggregatin Fan Out flw 01.ibm.cm/supprt/knwledgecenter/?lang=en#!/SSKM8N_8.0.0/cm.ibm.etls.mft.dc/ac1 2290_.htm IBM IIB Knwledge center Aggregatin Creating the aggregatin Fan In flw 01.ibm.cm/supprt/knwledgecenter/#!/SSKM8N_8.0.0/cm.ibm.etls.mft.dc/ac12300_.ht m IBM IIB Knwledge center Aggregatin Handling exceptins in aggregatin flws 01.ibm.cm/supprt/knwledgecenter/#!/SSKM8N_8.0.0/cm.ibm.etls.mft.dc/ac12340_.ht m IBM IIB Knwledge center Aggregatin Setting timeut values fr aggregatin 01.ibm.cm/supprt/knwledgecenter/?lang=en#!/SSKM8N_8.0.0/cm.ibm.etls.mft.dc/ac1 2320_.htm Page 12 f 13
IBM IIB Knwledge center Aggregatin Aviding thread starvatin in Fan In flws 01.ibm.cm/supprt/knwledgecenter/?lang=en#!/SSKM8N_8.0.0/cm.ibm.etls.mft.dc/ac1 2313_.htm IBM IIB Knwledge center Aggregatin Cnfiguring queues used fr Aggregatin 01.ibm.cm/supprt/knwledgecenter/?lang=en#!/SSKM8N_8.0.0/cm.ibm.etls.mft.dc/bc2 8130_.htm?cp=SSKM8N_8.0.0%2F1-7-7-0-3-0 IBM IIB Knwledge center Samples - Aggregatin 01.ibm.cm/supprt/knwledgecenter/#!/SSKM8N_8.0.0/cm.ibm.etls.mft.samples.aggregat in.dc/dc/verview.htm IBM develperwrks Extending Aggregatin http://www.ibm.cm/develperwrks/websphere/library/techarticles/1304_marie/1304_marie.html IBM develperwrks Dynamic Message Aggregatin http://www.ibm.cm/develperwrks/websphere/library/techarticles/1306_chakrabrty/1306 _chakrabrty.html Page 13 f 13