A Messaging-Based Integration Architecture for Print Production Workflow Systems

Transcription

1 A Messaging-Based Integration Architecture for Print Production Workflow Systems Claes Buckwalter Digital Media, ITN, Linköping University, Sweden Abstract A print production workflow consists of a number of systems specialized at executing specific tasks. In order to realize an efficient workflow it is not enough to simply assemble all the required systems, the systems must also communicate with each other and exchange information. The printing industry has used several approaches for integrating systems. A common approach has been to use so-called hot folders to exchange content files, such as PDF documents, between systems. Another approach, which focuses on the exchange of production and configuration data, is the ML-based Job Definition Format (JDF) and its messaging protocol Job Messaging Format (JMF). The need to integrate heterogeneous systems is nothing unique to the printing industry. The experience and knowledge in this area has been documented in several pattern languages. By applying a combination of several such pattern languages, this paper presents a hypothetical integration architecture that enables a system to communicate with other systems using JMF messaging. Introduction One prerequisite for efficient print production is that the systems involved Management Information Systems (MIS) and the software and hardware in prepress, press, and postpress can communicate and exchange information continuously during production. The data format and communication technology that is quickly becoming the de facto standard for solving this integration problem is Job Definition Format (JDF). The JDF Specification [5], which is maintained by the industry consortium CIP4 [7], defines an ML-based data format for specifying printed products and the workflow steps required to produce them. In addition, the JDF Specification defines Job Messaging Format (JMF), a messaging protocol for exchanging information between systems. The main objectives of JMF are transferring job data between systems, tracking jobs, tracking material consumption, and monitoring and controlling systems. The main design principles of JMF share similarities with traditional messaging integration technologies, such as -Oriented Middleware [9], and the Web Service [9] standards that emerged at approximately the same time as JMF. The experience and knowledge in the domain of system integration using messaging has been documented in several pattern languages [1, 2, 8, 9]. A pattern language is a collection of reusable solutions, patterns, that are applied to solve reoccurring problems in a specific domain for example the domain of messaging. Thus, when designing a JMF integration layer, the portion of a system that handles JMF interactions with other systems, there are several pattern languages where applicable well-proven solutions can be found. In this paper a hypothetical architecture for a JMF integration layer is presented expressed using pattern languages for system integration using messaging [1, 2, 8, 9]. Job Messaging Format (JMF) A JMF message is a small ML document that is exchanged between systems using two possible transport protocols: a proprietary file-based protocol that uses hot folders, or the standard network protocol HTTP [10]. Although the JDF Specification [5] defines both file-based and HTTP-based JMF messaging, in practice only HTTP is used. The JDF Specification implicitly favors HTTP and the CIP4 s Interoperability Conformance Specifications (ICS) [6] explicitly require HTTP-based JMF messaging. Therefore, this paper discusses JMF from the point of view of HTTP-based JMF messaging. JMF Families JMF messages are small ML documents. A JMF message consists of a message envelope in the form of a root ML element named JMF, which contains one or more messages that constitute the actual data. These messages are divided into six message families: messages modify the state of a system, for example submitting a new job to a RIP s queue. Query messages query the state of a system, for example querying the status of a printing press. Registration messages instruct a system to send messages to a third system. A MIS may for example tell a prepress system to send the plates it produces to a press. Response messages are synchronous replies to a, Query, Registration, Acknowledge, or Signal. Acknowledge messages are asynchronous replies to a, Query, or Registration Signal messages are events sent by a system when it reaches a certain state or when a specified interval of time has passed. For example, a press may notify a MIS each time 100 sheets have been printed. A Query is used to specify what Signals a system should generate. Figure 1 illustrates a message flow initiated by a Query or. Solid message lines correspond to a HTTP Request; dashed message lines correspond to a HTTP Response.

2 Table 1. JMF Families JMF Family HTTP Request HTTP Response Pattern Query Registration Response Document Acknowledge Document Figure 1. Example of a message flow initiated by a Query or Signal Event The messages, Query, and Registration are request messages that initiate a JMF conversation, they are always sent in a HTTP Request. Response, Acknowledge, and Signal messages are reply messages. However, only the Response message is sent in a HTTP Response. Both Acknowledge and Signal messages are sent in HTTP Requests. Messaging Patterns Systems in a print production workflow that communicate using JMF messaging must be capable of both initiating message interactions and participating in message interactions initiated by third parties. For example, an MIS submitting a job to a prepress system initiates a message interaction, which the prepress system must respond to. These two cases correspond to two fundamental message interaction patterns: the Send/Receive pattern and the Receive/Send pattern [2]. A system that initiates a JMF message conversation applies the Send/Receive [2] pattern. The system sends a request in the form of a, Query, or Registration message to a third party, and receives a synchronous reply in the form of a Response message. A system that receives a JMF request message from another system processes the JMF message and replies with the processing results in a Response message. This corresponds to the Receive/Send [2] pattern. In Figure 1 there are several occurrences of the Send/Receive and Receive/Send message interaction patterns: System A participates in a Send/Receive interaction; System B in both a Receive/Send and a Send/Receive interaction; and System C in a Receive/Send interaction. The six message families of JMF correspond to three message construction patterns documented in the pattern language of [9]. The pattern is used to invoke a procedure in another system. The Document pattern is used to transfer data between systems. The Event pattern is used to asynchronously notify systems of events. Table 1 lists the JMF message families, their message construction patterns and specifies whether they are transported in a HTTP Request or a HTTP Response. Coupling An important concept in software design is coupling. Coupling is a measure of how dependent components in a system are on each other. Systems consisting of tightly coupled components are often difficult to modify because a modification to one component in the system affects the other components that it is tightly coupled to. Therefore, it is generally considered good practice to design systems with as loosely coupled, decoupled, components as possible. The notion of coupling can also be applied to system integration using messaging. Eugster et al. [8] have identified three dimensions of (de-)coupling in the area of system integration using messaging: Time the sender and receiver of a message does not need to be active at the same time Space a message is sent to a symbolic address representing the target system, not directly to a target system Synchronization senders do not block while sending a message and receivers are notified by a callback when a new message is available These dimensions of have been formalized by Aldred et al. [1] and used by Buckwalter [3] to classify the two communication models of JMF. This classification is summarized in Table 2. Besides the properties of file-based and HTTP-based JMF messaging, Table 2 also specifies the desired properties of a JMF integration architecture that can be added to a system in order to allow it to communicate using JMF messaging. The remainder of this paper discusses the design of a hypothetical JMF integration architecture, integration layer, which exhibits these properties.

3 Table 2. Decoupling properties of JMF Time Space Synchronization The Send/Receive message flow begins with a component in a system sending a message. The proposed hypothetical JMF integration layer s Send/Receive message flow consists of the following steps: File-based JMF Yes Yes HTTP-based JMF JMF Integration Layer No Yes No Yes JMF Integration Architecture Blocking send Blocking receive Blocking send Non-blocking receive Non-blocking send Non-blocking receive Any system MIS, prepress workflow, offset press, digital press, cutting machine that wants to communicate using JMF must implement some form of software layer that handles the sending, receiving, and processing of JMF messages. Figure 2 illustrates such a layer. A well-designed JMF integration layer would shield a system s internal components from the details of sending and receiving JMF messages and would provide a high degree of between a system s components and the systems that they communicate with using JMF messaging. The approach proposed by this paper is to implement the JMF integration layer as a messaging architecture. The proposed architecture is described using the pattern language of Hohpe and Woolf [9]. This pattern language provides the building blocks to describe the different processing steps required of a JMF integration layer. The pattern language also includes graphical symbols that represent its patterns. These symbols are used to visualize the proposed JMF integration architecture. Figure 2. JMF Integration Layer The proposed JMF integration architecture is based on the concept of Pipes and Filters [9]. Pipes, realized by Channels [9] or queues, are used to decouple components and the various processing steps, filters, in the JMF integration architecture. Another central concept is that of message routing. Based on a message s properties or some external stimuli, the message is routed through various processing steps before it is delivered to its final destination an external system or a component belonging to the system implementing the JMF integration layer. A system s JMF integration layer can be divided into two subarchitectures: one for handling the Send/Receive [2] message interactions of the system s components; and one for handling the Receive/Send [2] message interactions of the system s components. Figure 3 shows the architecture of the proposed JMF integration layer using the symbols of pattern language [9]. Although both Send/Receive and Receive/Send message flows are shown in Figure 3 this article focuses on the Send/Receive message flow, the symbols inside the dashed rectangle. 1. A component sends a, Query, or Registration by placing the message on an outbound queue, a Channel [9], which is shared by all components sending messages. Each message put on the queue is equipped with a unique message ID, the destination address where the message is to be delivered, and the Return Address [9] of the component sending the message. 2. A Smart Proxy [9] picks the message off the outbound queue and stores the message s unique ID and the return address of the component that sent the message. The message ID is used as part of the Correlation Identifier [9] pattern that will later allow the proxy to route the message s reply to the return address of the component. 3. The proxy forwards the message to an Envelope Wrapper [9] that wraps the, Query or Registration in a JMF envelope, thereby constructing a complete and valid JMF message. The envelope wrapper then puts the JMF message on an outbound delivery queue. 4. A HTTP client picks the JMF message off the outbound delivery queue and sends it in a HTTP Request to the message s destination address the URL of the system that the message should be sent to. The JMF integration layer has now completed the delivery of the message on behalf of the sending component. If the HTTP client fails to deliver the message to the destination address due to network problems, system downtime, or whatever other reason, it retries for a specified number of times or during a specified time period. If the JMF message still cannot be delivered it is placed on a Dead Letter Channel [9] where a system administrator can examine the message and take necessary actions. When the system that the message was delivered to has processed the message it sends a synchronous reply containing the processing results back to the HTTP client in a HTTP Response. The HTTP client extracts the JMF message from the HTTP Response and places the JMF message on an inbound JMF message queue for the JMF integration layer to deliver the reply back to the component that sent the original request. 5. A special message processor that verifies the correctness of inbound JMF messages picks the message of the queue. If the message is invalid it is placed on an Invalid Channel [9] where a system administrator can manually analyze the message to determine the fault. If the JMF message is valid it is forwarded to an unwrapper part of the Envelope Wrapper [9] pattern which extracts the JMF Response message and forwards the Response to the Smart Proxy [9]. 6. The Response message contains the ID of the request that it is a reply for the Correlation Identifier [9] pattern. Using the request s ID the Smart Proxy [9] can lookup the stored Return Address [9] of the component that sent the request. The proxy then places the Response on the components inbound Channel [9].

4 Figure 3. Receive/Send (top) and Send/Receive (bottom) message flows through the JMF integration layer 7. The component that sent the original request (, Query, or Registration) message is notified when a Response message is available on its inbound queue. The component removes the Response message from the queue, correlates it with the corresponding request message by extracting the request message s ID from the reply Correlation Identifier [9] pattern and then processes the reply. The JMF integration layer has completed delivering the reply message back to the component that sent the corresponding original request message. Conclusions By implementing a JMF integration layer a system in the print production workflow be it an MIS or software/hardware in prepress, press, or postpress can completely decouple it s internal components from the details of the message interactions with external systems. The only action required of a system component in order to send a message is to place the message together with delivery instructions on an outbound queue and wait to be notified when a reply is available on its inbound queue. The component sending the message does not connect directly to the message s destination system and is not dependent on the destination system being available at the time the message is sent the component is space decoupled and time decoupled from the destination system. The sending component is also synchronization decoupled from the destination system of the message. The sending component s thread of execution returns as soon as the message has been placed on the JMF integration layer s outbound message queue; and when a reply message is available on the component s inbound message queue the component is notified by a callback. The principle of a messaging integration layer is nothing unique to JMF integration in the printing industry. Several other industries use similar approaches to decouple system components from the details of message interactions with external systems. In recent years, the concept of an Enterprise Service Bus (ESB) [4], a configurable infrastructure that facilitates message processing, routing, and delivery, has become increasingly popular. In theory, the entire JMF integration layer presented in this paper could be realized with an Enterprise Service Bus. References 1. L. Aldred, W. M.P. van der Aalst, M. Dumas, and A.H.M. ter Hofstede. On the Notion of Coupling in Communication Middleware. Lecture Notes in Computer Science, Volume 3761, pp Springer Verlag, A. Barros and E. Boerger. A Compositional Framework for Service Interaction Patterns and Interaction Flows. Proceedings of the Seventh International Conference on Formal Engineering Methods (ICFEM'2005), Manchester, UK, pp Springer Verlag, C. Buckwalter. Integration Patterns Applied to Print Production. TAGA 2006 Proceedings, D. Chappell. Enterprise Service Bus. O Reilly Media Inc, 2004.

5 5. CIP4. JDF Specification Release 1.3, accessed March CIP4. Interoperability Conformance Specification (ICS) Registry. accessed March CIP4. The International Cooperation for the Integration of Processes in Prepress, Press and Postpress Organization (CIP4) home page. accessed March P.Th. Eugster, P.A. Felber, R. Guerraoui, and A.-M. Kermarrec. The Many Faces of Publish/Subscribe. ACM Computing Surveys, 35(2), pp June G. Hophe and B. Woolf. Enterprise Integration Patterns. Addison-Wesley, World Wide Web Consortium. Hypertext Transfer Protocol (HTTP). accessed March Author Biography Claes Buckwalter has a M.Sc. in Media Technology and Engineering from Linköping University, Sweden, where he is currently a Ph.D. student. Buckwalter s research interests are integration technologies and workflow automation applied to the graphic arts industry. Buckwalter is also involved in the CIP4 organization where he is currently chairman of the Tools & Infrastructure working group.