TIBCO Complex Event Processing Evaluation Guide

Transcription

1 TIBCO Complex Event Processing Evaluation Guide This document provides a guide to evaluating CEP technologies. Global Headquarters 3303 Hillview Avenue Palo Alto, CA Tel: Toll Free: Fax: , TIBCO Software Inc. All rights reserved. TIBCO, the TIBCO logo, The Power of Now, and TIBCO Software are trademarks or registered trademarks of TIBCO Software Inc. in the United States and/or other countries. All other product and company names and marks mentioned in this document are the property of their respective owners and are mentioned for identification purposes only. This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. This document is provided for informational purposes only and its contents are subject to change without notice. TIBCO makes no warranties, express or implied, in or relating to this document or any information in it, including, without limitation, that this document, or any information in it, is error-free or meets any conditions of merchantability or fitness for a particular purpose. This document may not be reproduced or transmitted in any form or by any means without our prior written permission. XX/06 Version 1.0

2 An Introduction to Complex Event Processing Complex Event Processing (CEP) is a set of technologies that processes complex events or event aggregates and event relationships to other events or business entities. CEP allows "events" to be processed on a continuous, asynchronous basis. CEP applications can usually be described as providing situation awareness, sense and respond, or track and trace, any of which can provide high value and utility to many business scenarios. Situation awareness is about knowing" the state of the product, person, document, or other entity of interest, at any point in time. Obtaining this knowledge of state requires knowledge of possible and priorr states, and the continuous monitoring of relevant events that might affect the current state. As an example, a dashboard indicates all performance indicators for a runtime production process: all the production plant events are monitored and the situation, or health, of the production process is determined via performance indicators that are shown in real-time to one or more operators. Sense and respond is about detecting some significant fact about the product, person, document or other entity of interest, and responding accordingly. To achieve this result the software does the following: Monitors events that indicate what is happening to this entity. Detects when something significant occurs. Executes the required response. As an example, a system monitoring cell phone or credit card usage may detect cell phone or credit card (cardholder present) consecutive transactionss at different locations that cannot be accounted for by personal travel. Detection of such transactions indicates that an act of fraud is in progress. The system responds accordingly, denying the transactions, and invoking the necessary business process to handle the situation as defined in standard operating procedures. Track and trace is about tracking the product, person, document or entity of interest over time and tracing pertinent facts like location, owner, or general status. An example would be tracking events from an RFID-enabled inventory control system where at any point in time you need to know how many widgets are in what location. Situation awareness," sense and respond," and track and trace" can all be classified as types of activity monitoring, for which the continuous evaluation of incoming events and their processing in real-time is highly desirable. For this reason, CEP is often described as a technology for Business Activity Monitoring (BAM), although more technical domains such as system monitoring are also applicable, and CEP also allows for direct response actions as well as the act of monitoring. Technical Requirements of a CEP System Complex Event Processing systems must be able to (1) receive and record events, (2) identify patterns of events over time and space, including the non-occurrence of events, and (3) respond with appropriate events, decisions, and/or actions. CEP tools typically provide features around the following: Some event / event object representation, with associated event channels (for different event mechanisms) and destinations (for different types of events). 2

3 One or more pattern detection operators or algorithms or languages, targeting different types of inter-event relationships and various levels of throughput and latency. Some mechanism, implicit or explicit, for storing state across events patterns and time. Language and Model-Driven Approaches to CEP Event processing platforms that provide CEP capabilities can exploit a variety of event pattern representation mechanisms that can variously be classed as business models, software models, and languages. Procedural vs. Declarative CEP tools can use either or both of the 2 main styles of computer operators: procedural or ordered models and code, such as script languages or orchestration diagrams, or declarative models and code like rules. Generally the latter can be more difficult to understand but can be easier to maintain (for example: new rules can be added without knowing where to put them). Event and Object Management and Persistence Example Models used in CEP Typically CEP tools will allow multiple storage options, varying from in-memory for performance to databases for reliability. One middle route is to use distributed in-memory data grids or caches to provide data scalability with performance and reliability. 3

4 Evaluation Guide There are various use cases for CEP technology which have different characteristics. The below list provides a typical list of features and functions to allow CEP systems to be compared. CEP Requirement Feature Detail Notes A1 Event Channels Standard Channels A2 JMS HTTP A3 A4 Proprietary Channels RV MQ A5 Other An example might be TIBCO FTL A6 A7 Domain Channels Custom Channels Industry-specific channels or adapters Custom channel type required B1 Graphical Models Event Models Input for definition Event types, inheritance etc relationships and attributes; may be UML-based B2 B3 B4 B5 B6 B7 B8 Output or report Event Object Models Input for definition Object types, inheritance etc relationships and attributes; may be UML-based Output or report Entity State Models Input for definition States and state transitions; may be UML-based Output or report Process Models Input for definition Sequences of event processing operations (or Event Processing Network); may be UML-based or BPMN-based Output or report 4

5 C1 Language Operations Rules Event-Condition-Action Declarative; for unordered event clouds; actions can be events. C2 Event-Inference Rules Declarative; for unordered event clouds; actions can be events or automatic inferences. C3 Queries Continuous Queries Can be declarative; for ordered event streams; can feed rules or processes. C4 Explicit Patterns Temporal regular expression logic Textual description of rule patterns C5 Algorithms Scripts and Code Procedural mechanisms for defining particular algorithms etc C6 Analytics and Statistical Operators Specialist operations required not covered above May be specialist temporal/spatial operators or for pattern discovery / predictive analytics C7 Other Context support Mechanisms for global values like scorecards May also be covered by state etc references C8 Extensibility Language extension mechanisms Custom functions, algorithms, analytics etc. D1 D2 Event Augmentation Database Access JDBC data access CRUD etc operations Custom data access Domain-specific data stores or access mechanisms D3 D4 Service Access Integration with SOA May be via events, WSDL, etc Process Access Integration with workflows May be via events E1 E2 Stateful processing and management Single process or operator In-memory Database Requires augmenting for high availability Allows for disaster recovery E3 Multi-process or Distributed memory / grid Can enable high availability E4 operator Combination Could be persisted data grid, support distributed event processing 5

6 F1 F2 Operations Support Distributed processing Distributed for throughput Distributed for pipelined event processing Needs to support event recovery Needs to support agent recovery F3 Distributed event / event object state storage F4 Failover and Hot standby capability F5 Resilience Failover support F6 Hot deploy Operator updates as event payloads F7 Engine auto-restart on reload F8 Performance Preprocessor, etc Performance shortcuts. optimizations F9 Operations monitoring Dashboard for monitoring CEP execution` F10 OS and platforms Development platform IDE, business UI, etc required F11 Deployment platform Runtime server / engine G1 Development Environment Integrated Development Eclipse or other Preferred IDE or modeling tool Environment G2 G3 Verification tools Logic verification Graph network completeness checks, rule logic completeness, diagram outputs etc Validation tools Unit test of operations Test an operation against an event or situation G4 G5 Batch test of operations Simulation operations Test an application or subset against a set of tests Run an event simulator to stress test and obtain operation statistics G6 Debugging Local debugging Stepwise debugging etc G7 Remote debugging Local viewing of a remote engine operations G8 Team access Project Repository support Allow team development via SCCS 6

7 H1 Business interface End-user editors Event pattern UI Update rules / queries / patterns in a business UI H2 Decision and Action UI Update event responses eg decision rules / tables etc in a business UI H3 Dashboard metrics and screen designer Business user layout of results dashboard against KPIs H4 Application Lifecycle Management Workflow for controlling edits to CEP Control application for business users H5 Event Dashboard UI for monitoring events, KPIs May have a whole set of additional requirements eg security, widgets etc I1 Project Considerations CEP use cases Related domain / industry use cases I2 Related technology, scale use cases I3 Development partners and SIs Initial developer assistance or turnkey development 7

8 Using the Evaluation Guide Organizations should use the above Guide as just that a guide and customize it to suit their requirements. For example, an algorithmic trading application may want to extend the language analysis into specific temporal operators required for the sorts of trading anticipated. A risk compliance application may want to specify certain performance throughput metrics for a known case study. A telco service policy application may need to refer to interfaces to particular packet inspection hardware. And so on. In addition to the customized evaluation form, organizations should add some scoring evaluation metric: for example a weight per feature, and an associated score per tool. Note that: Application Use Cases can often be readily mapped into event processing operations, except for particularly complex event aggregations which may instead be described separately as event patterns (alongside any associated business rules). Such use cases are invaluable for extracting required tool details for CEP applications, and can replace or augment the model / language sections in the Guide. Some features are interchangeable; for example, CRUD operations on complex events via a query language can almost always also be accomplished by rules, with more or less development effort, and often vice versa. There are not many (or indeed i any) CEP-specific standards at present; other IT standards such as UML (Class, State, and Production Rules), BPMN, and derivatives of SQL are often used in CEP. If standards are considered important, consider a requirement for standards participation in your CEP selection questions! For vendor-supplied responses, particular care should be taken to ensure that responses are comparable. For example, features can be out of the box supported (often with an accompanying example) and configured by a UI or files, to API supported, to requiring custom extensions. A good example of this is distributed cache support which most vendors can potentially support, but few do out of the box. Feedback To provide feedback and suggestions for future versions of this document, please comment on either of the following TIBCO CEP blog posts evaluation-check-list/ Future updates to this will likely be advertised on the TIBCO CEP blog. 8