Complex Event Processing in a High Transaction Enterprise POS System

Transcription

1 Complex Event Processing in a High Transaction Enterprise POS System Samuel Collins* Roy George** *InComm, Atlanta, GA **Department of Computer and Information Science, Clark Atlanta University, Atlanta, GA {scollins@incomm.com, rgeorge@cau.edu} Abstract Complex Event Processing (CEP) is a software methodology that uses the concept of an event for system development. CEP has several advantages that make it suitable for time- and resource-constrained problems. In this research we develop an abstract CEP architecture to support a high transaction enterprise Point Of Sale system. The enterprise CEP provides a highly available and reconfigurable environment for an industrial application. Results from the deployment of the system show that the CEP application provides a scalable solution to the problem of high transaction environments. I. Introduction Complex Event Processing (CEP) [Luc2002] is a methodology for processing multiple events that occurs in an organization by indentifying the relevant events, and acting on them, based on the notion of the event as the organizing paradigm. CEP has been used as a solution strategy for applications in transportation [GSS2008], trading, environmental monitoring [KYK2008], healthcare [WLG2005], RFID-based asset tracking, etc. [EN2010]. CEP has several advantages that make it suitable for time- and resource-constrained problems [ZCM2005] seen in enterprise applications. Large transaction processing applications place heavy demands on the software infrastructure. First, the volume of transactions can overwhelm the capacity of the system to respond, leading to software failures. Second, the notion of the event considers the temporal component of the data stream which are handled organically by CEP systems using windowing operations in SQL-like aggregation or joins to correlate events in time. Finally, while stream imperfection such as out of order packets and blocking I/O, can introduce expensive overhead to conventional systems, CEP provides the ability to incorporate feedback systems, and dynamically adjusted timeouts to address this problem. The characteristics of CEP systems include the ability to detect simple event patterns in streaming event data, compose complex events, and react to complex event patterns in a highly efficient and optimized manner [GO2003, Owe2007]. This paper develops details the application of CEP in the enterprise architecture of a high transaction Point of Sale (POS) environment. The CEP component provides the benefits of high availability and optimized system performance, in addition to the advantages described previously. The paper is structured as follows: Section II outlines previous efforts in Complex Event Processing and describes the abstract architecture used in this project. Section III details a real world example and shows how the CEP is used to enhance system performance. The conclusions and future work are presented in Section IV. II. Background The CEP approach used in this paper is based on the notion of events used to model data in the context of enterprise applications. An event is an occurrence or happening of significance that can be defined as a region or collection of regions in spatial-temporal-attribute space. The event data type based on the following mandatory attributes: eid, space, time, event-name, event-granularity, event-associations, and event-topic. The attribute space provides the geographic information, abstract symbolic information like the merchant ID, processor errors, failures and transaction volume to locate the event. The timestamp may include start and end times for an event or be a point in time.

2 In event processing, discrete events have orderly relationships that aid the generation of hypotheses, guides understanding, helps and control actions. The following relationships between events are considered. Temporal relationship. In the event structure, the fundamental element is the event which is then related to other events by a temporal ordering such that events occur in a particular sequence, Allen s temporal interval algebra [All1983] for representing and reasoning about temporal relationships may be applied for presenting the temporal relationship between events. Thirteen relationships including seven basic relations and their symmetrical relations are represented in Allen s temporal algebra. Spatial relationship. Spatial relations are classified into two types: topological relations that describe neighborhood and incidence (e.g. overlap, disjoin) and directional relations that describe order in space (e.g. south, northwest). A general spatial relationship considers 12 directional relationships and 6 topological relationships [LOS1996]. The directional relationships are classified into the following three categories: strict directional relations, mixed directional relations, and positional relations. The topological relationships include equal, inside, cover, overlap, touch, and disjoint. CEP Architecture Figure 1 shows the abstract complex event processing architecture used in this research effort. Streaming data is converted into contextualized event and handled in real time by the Business Rule Engine (BRE). The meta-rule interpreter is used to translate the high level rules defined in the system to the object rule engine. This open architecture permits any object rule engine to be deployed, the only requirement is that the mapping between the meta-rules and the native rule structure of the object rule engine to be defined. The main memory data repository and the contextbase support the mapping from low-level event to semantic domain event. Spatial and temporal reasoning are provided by the BRE. Fig. 1. Complex Event Processing Architecture CEP implementations include general purpose stream management systems such as Aurora [ACC2003] and its second generation version Borealis [AAB2005]; the general purpose complex event processing system, Cayuga [DGP2007]; STREAM, with integrated stream and storage data capability [ABB2004]; and TelegraphCQ, a continuous query system [KCC2003]. Commercially available systems include StreamBase, Progress, Coral8, and TIBCO. The applications for Complex Event Systems are numerous and do not all fit the standardized approaches offered by current implementations [SC2005, EN2010]. The CEP approach presented here is designed to overcome problems seen in standard CEP systems. Current deployments are large and centralized, and are not compatible with the mobile and decentralized approach to decision making currently under way. The ability to embed CEP properties on existing hardware cannot be easily achieved with existing implementations. A second and more significant drawback is that most CEP systems are based on the database paradigm. Data streams in enterprise POS applications can change rapidly; however, the CEP approach forces the data stream to conform to schema creation and usage paradigms. Detection of events is primarily based on the ability to perform widow-based joins. Joins are

3 expensive database operations, and special indexing structures are needed to make them efficient (which further increases system footprint). Finally, events in data streams can be of different periodicity, with some having relatively long periods. Database approaches handle these using long transactions which have poor efficiencies. The system presented in this paper uses an integrated rule/data based approach to overcome these problems. III Complex Event Processing for POS Transaction Processing The CEP solution was implemented for the industry leading distributor of stored-value and prepaid cards which processes several hundred million transactions every year. In a stored-value card, the monetary value is physically stored on the card and not in an externally recorded account. The software infrastructure has to provide flexible transaction processing and activation options to process transactions. The enterprise architecture supports several modes of transaction processing, with each method supporting activation, refunds, and deactivation. In addition it supports web-based applications that enable the sale of services including cable and satellite television, wireless and more from providers. Figure 2 shows the CEP software architecture deployed. The Enterprise POS Model During normal processing, Real Time events are received from the Merchant, logged, and then sent to the Provider for processing. The Provider s response is then sent by the system to the Merchant to complete the transaction. In this normal situation, the transaction proceeds through the Merchant Interface Layer (MIL), the Application Processing Service (APS), and the Service Provider Interface Layer (SPIL) in the system. Transaction volumes are normally range from high to very high levels; the latter being associated with national and regional holidays (corresponding to high consumer gift giving) The transaction mode heterogeneity and the volume of transactions puts a very heavy load on the software infrastructure resulting in exceptions which cause the non-completed transactions. We define a Store and Forward model based on Complex Event Processing (Front End SAFing) which can alleviate the problems of software exceptions, which include the following: 1. Stored value cards in an incorrect state, 2. Discrepancies in billing references, and 3. FES limitations. The solution inserts a CEP component between the MIL and the APS to respond to the Merchant in the event that the APS is not able to complete the transaction. This component is based on the Store and Forward (SAF) model which stores transactions that cannot be completed until a time that they can be completed. The SAFing model is applied to the beginning of the transaction process and is therefore referred to as Front End SAFing (FES). FES contains a Merchant Parsing Layer (MPL) to translate the transaction. Merchant MIL M T L APS SPL Provider InComm Fig. 2: Real Time Transaction Model Merchant MIL M P L FES M T L APS SPL Provider InComm Fig. 3: FES Transaction Model The SAF service enables the transaction processing engine to deliver messages reliably between distributed applications (such as external APIs for vendors/service providers). If the destination is not available at the moment

4 the messages are sent, either because of the network problems or system failures, the messages are saved to a database, and are forwarded to the remote endpoint once it becomes available. The transaction processing engine utilizes the SAF service to enable the APS component to reliably send messages through the SPIL component to a remote transaction endpoint. FES has three modes of operation. In the first FES mode, FES operates transparently between the MIL and the APS. FES parses and logs the transaction, then passes the request to the APS to continue the normal operational process. The APS will pass the response to FES where it will be logged into the FES database before transitioning to the MIL. In the second phase of FES, in addition to parsing and logging the transaction, FES will send a preconfigured response for the transaction to the MIL as show below. 1 MIL FES APS Card Request FES Response APS Response 123 Act Success null 345 Inquiry Failure null... Xyz Deact Success null 4 Database FESQP 2, 3 1. FES receives transaction from the MIL 2. FES parses and logs the request 3. FES responds to the transaction and logs response 4. FES sends the response to the MIL Fig. 4 FES Queuing Operation When FES is operational, there is no validation. Based on the Merchant configuration, the same response is sent for all transactions. The third phase of FES operation begins once the APS is back online. At this time, three separate processes operate concurrently. First, once the APS is back on line, the FES Queue Processor begins to clear the SAFed transactions by forwarding them to the APS for processing. The APS response is logged in the FES database. The APS response is not sent to the MIL, since FES previously sent a response to the Merchant for the transaction. If the APS response does not match the FES response, the discrepancy is noted in the FES database for exception reporting. Second, the FES parses and logs the transaction received from the MIL, and then verifies that there are no previous transactions for the card in the FES queue. Once it is determined that there are no SAFed transactions for the card, the transaction passes to the APS for processing. The response is then logged and passed to the MIL. Third, if a transaction was stored during FES operation, and while the FES Queue Processor (FESQP) clears the first transaction, if another transaction is received for the same card: the new transaction is then stored sequentially in the FES database and the configured response is returned to the MIL. The FESQP will process the transactions in the order in which they were received. Sample FES rules are shown in Figure 3- note that the operational mode is determined by the rule. It determines whether the transaction should proceed normally. The operational mode is an indicator of the number of exceptions generated by a particular merchant-vendor combination. CONFIGURATION CHANGE INFORMATION Configuration FES.CONDITION.XYZ Previous Value New Value Operational Mode Failure Count 1 YELLOW GREEN ORANGE Total Transactions 14 Percentage of Failures 7.14% Failure Samples Card Number Response Code Source

5 Rule Implemented IF MERCHANT=XYZ AND SIZE=MEDIUM AND CURRENTMODE=YELLOW AND FAILURES <= 6 AND FAILURES PERCENT <=10.0 AND DEFAULT MODE IN ( NONE,GREEN) AND ELAPSEDTIMESLICES >=2 AND TIMESLICE =188 THEN CHANGE MODE TO GREEN Figure 3: Sample FES Rules: Changing the Operational Mode The deployment of FES has been critical in ensuring the high availability in the enterprise architecture. As noted previously the volume of transactions and the heterogeneity of origin can stress the enterprise POS architecture. As an example, during a high transaction scenario, Front End SAFing (FES) was enabled for several Merchants. During the time that FES was enabled the FES application intelligently SAFed 278,725 transactions. Of the 278,725 transactions, 5,424 (1.95%) did not result in a successful response code (0 or 136) from the APS layer. 76% of the failed transactions resulted from improper usage modes. IV Conclusions Complex Event Processing provides a good solution to the problems of high transaction environments. In this paper we presented a unique application of CEP in a highly stressful industrial environment. In this environment, dealing with the activation and redemption of stored value cards, a highly available, high performance enterprise infrastructure is vital to the organization. This effort reported on the details of the architecture of the CEP system, and the deployment environment. Currently we are considering enhancing this system to also to detect fraudulent card activation and usage since all transactions are currently being handled by the FES component. The performance characteristics of the FES component would permit fraudulent transactions to be detected in near real time. Acknowledgements This research is supported in part by the US Army TEC Contract W9132V-09-P The content of this work does not reflect the position or policy of the sponsors and no official endorsement should be inferred. V References [AAB2005] Abadi, D., Ahmad, Y., Balazinska, M, Cetintemel, U., Cherniack, M., Hwang, J., Linder, W., Maskey, A., Rasin, A., Ryvkina, E., Tatbul, N., Xing, Y., Zdonik, S., The Design of the Borealis Stream Processing Engine, in Proceeding of the Second CIDR Conference, January [ABB2004] Arasu, A. and Babcock, B. and Babu, S. and Cieslewicz, J. and Datar, M. and Ito, K. and Motwani, R and Srivastava, U.and Widom, J., STREAM: The Stanford Data Stream Management System, Technical Report. Stanford InfoLab, 2004 [ACC2003] Abadi, D., Carney, D., Cetintemel, U., Cherniack, M., Convey, C., Erwin, C., Galvez, E., Hatoun, M., Maskey, A., Rasin, A., Singer, A., Stonebraker, M., Tatbul, N., Xing, Y., Yan, R., Zdonik, S., Aurora: A Data Stream Management System, in Proc ACM SIGMOD International Conference on Management of Data, San Diego, California, 2003, page 666. [All1983] Allen, J.F. Maintaining knowledge about temporal intervals, Communications of the ACM, vol. 26, pp , [DGP2007] Demers, A., Gehrke, J., Panda, B., Riedewald, M., Sharma, V., and White, W., Cayuga: A General Purpose Event Monitoring System, Proceedings of the Third Biennial Conference on Innovative Data Systems Research (CIDR 2007), Asolimar, California, January [EN2010] Etzion, O., and Niblett, P., Event Processing in Action, Manning Press, Stamford, CT, 2010 [GO2003] Gloab, L., and Ozsu, M., Issues in Data Stream Management, SIGMOD Record, Vol. 32, No. 2, pages 5-14, June [GSW2005] George, R., Shujaee, K., Wu, B., and Liu, Z., "ewellness: Building a Smart Hospital by Leveraging RFID Networks," Proc. of the 27th IEEE Engineering in Medicine and Biology Society (EMBS2005), Sep [GSS2008] George, R., Shujaee, K., Sazegarnejad, M., and Rogers, J., Distributed Decision Making in a Sensor Enabled Environment, International Conference on Sensing a Changing World, Nov.19-21, 2008, Holland. [KCC2003] Krishnamurthy, S., Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M., Hellerstein, J., Hong, W., Madden, A., Reiss, F., Shah, M., TelegraphCQ: An Architectural Status Report, IEEE Data Engineering Bulletin, Vol. 26, Issue 1, 2003, pages

6 [KYK2008] [Luc2002] [LOS1996] [Owe2007] [SC2005] ZCM2005] Korpeoglu, B., Yazici, A., Korpeoglu, I., and George, R., A New Approach for Information Processing in Wireless Sensor Network Database Applications, 2nd IEEE International Workshop on Networking Meets Databases (NetDB'06), Atlanta, GA, April Luckham, David, The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems, Pearson Education, Inc., 2002 Li, J.Z., Ozsu, T., and Szafron, D., Modeling of video spatial relationships in an object database management system, In Proceedings of the 1996 International Workshop on Multi-Media Database Management Systems, pages 124, Aug Owens, T. J., Survey of Event Processing, Technical Report, Air Force Research Laboratory, Information Directorate, Dec URL: Stonebraker, M., and Çetintemel, U., One Size Fits All: An Idea Whose Time Has Come and Gone, pp.2-11, 21st International Conference on Data Engineering (ICDE'05), 2005 Zdonik, S., Çetintemel, U., and Stonebraker, M., "The 8 requirements of real-time stream processing, " ACM SIGMOD Record, December 2005: