The Standards Based Integration Company Systems Integration Specialists Company, Inc. Naming & Design Requirements (NDR) CIM University San Francisco October 11, 2010 Margaret Goodrich, Manager, Systems Engineering SISCO, Inc. 6605 19½ Mile Road Sterling Heights, MI 48314 USA Tel: +1-903-477-7176 Fax: +1-903-489-0063 E-Mail: margaret@sisconet.com Topics: Purpose System Context Notation and Terminology Mappings References Edits and Change Requests 2
Purpose NDR will be coming out WG 19 IEC 62361-100 Currently in CD form Describes Mapping from CIM Profiles to W3C XML Schemas Mapping Facilitates Information Exchange of XML Documents with IEC CIM Semantics and W3C XML Schema Syntax. Define the rules needed to create XSDs that comply with both W3C and UN/CEFACT specifications 3 System Context & Profile Definition Map Contextual Model or CIM Profile to W3C XML Schema Contextual Model Concept is Borrowed From UN/CEFACT In CIM, Contextual Model is Referred to as a CIM profile or, more simply, a Profile A Profile is a Restricted Subset of the CIM A Profile is used to define the semantics of a single type of message encoded in XML Mapping Determines: A single, standalone XML schema (XSD) for a given contextual model (profile). Indirectly, the syntax of the instance XML documents to be exchanged. The relationship between definitions in the XML schema and definitions in the contextual model. Indirectly, the relationship between elements in the XML documents exchanged and the definitions in the CIM. 4
Notation and Termination Provides Definitions for the following Contextual Model Concepts: Classes: Structured, Concrete, Abstract, Root, Anonymous, Compound, and Enumeration Types: Simple and Basic; Quantity (as a simple type) Union: union members are structured classes Properties: Object, By-Reference, Compound and Simple SuperClass, Subclass, Categorized Documentation, Stereotype and Documentation XML Schema Mappings Use XML Representation Summary Notation Notation consists of an XML Schema construct with the following conventions (not all are listed here): Mandatory attributes are shown in bold, e.g. count Optional attributes are shown in standard, e.g. size Literal attribute values are shown in italics e.g. medium 5 Mappings A single CIM profile is mapped to a single XML Schema with the following form: <xs:schema targetnamespace = namespace-uri elementformdefault = qualified attributeformdefault = unqualified version = version-string > Content: (annotation, envelope-elem, envelope-type, (complex-type simple-type enum-type)*) </xs:schema> The envelope-elem is the single top-level element definition in the schema, as follows: <xs:element name= envelope-name type= envelope-name> </xs:element> 6
Mappings The envelope-type is the type definition for the envelope as follows: <xs:complextype name = envelope-name> <xs:sequence> Content: (root-elem*) </xs:sequence> </xs:complextype> Each root-elem is defined as follows: <xs:element name = root-name type = root-name minoccurs = min maxoccurs = max> </xs:element> 7 Mappings Each abstract or concrete class in the profile, including each root class, and each compound class is mapped to a complex-type definition as follows: <xs:complextype name = class-name abstract = abstract-value sawsdl:modelreference = class-ref> Content: (annotation, (whole-class derived-class)) </xs:complextype> 8
References Normative References Extensible Markup Language (XML) 1.1 (Second Edition)W3C Recommendation 16 August 2006 Namespaces in XML 1.1 (Second Edition) W3C Recommendation 16 August 2006 XML Schema Part 1: Structures Second Edition W3C Recommendation 28 October 2004 XML Schema Part 2: Datatypes Second Edition W3C Recommendation 28 October 2004 Semantic Annotations for WSDL and XML Schema W3C Recommendation 28 August 2007 Non-Normative References OWL Web Ontology Language Reference W3C Recommendation 10 February 2004 Unified Modeling Language (UML) Specification V2.2, Object Management Group UN/CEFACT XML Naming and Design Rules Version 3.0 Implementation Verification, 30 January 2009 9 Edits and Change Requests R06.1, R06.2 & R06.3 R06.1 provides for some general edits that are proposed R06.2 adds all changes from R06.1 plus DataTypes as a Structured Class R06.3 adds all changes from R0.1 and R06.2 plus the DataType Class Property 10
The Standards Based Integration Company Systems Integration Specialists Company, Inc. 61968 1-1: ESB Implementation Profile CIM University San Francisco October 11, 2010 Margaret Goodrich, Manager, Systems Engineering SISCO, Inc. 6605 19½ Mile Road Sterling Heights, MI 48314 USA Tel: +1-903-477-7176 Fax: +1-903-489-0063 E-Mail: margaret@sisconet.com Introduction Scope/Purpose Some Definitions Use Case Sequence Diagrams Integration Patterns Message Flows 12
Scope/Purpose To Define an Implementation Profile for integration using an ESB that supports both JMS and Web Services. This standard defines how message payloads are conveyed using Web Services and the Java Message Service (JMS) The Goal is to provide sufficient information to enable implementations to be interoperable. Will be used as an integral part of the interoperability tests for distribution management systems in 2009 and beyond. 13 Some Definitions Enterprise Service Bus (ESB) refers to a software architecture construct that provides foundational services via an event-driven and standards-based messaging engine. Java Message Service (JMS) - is an API that supports request/reply, publish/subscribe, and point-to-point messaging patterns. Service Oriented Architecture (SOA) is an architectural style for creating and using business processes, packaged as services. Simple Object Access Protocol (SOAP) is a standard that defines the formatting of XML messages and serves as a foundation layer of the Web Services Protocol stack. 14
Some Definitions - Continued Web Services Definition Language (WSDL) is an XML-based language that is used to describe Web Services. WSDL is used in combination with SOAP or XML schema to provide Web Services over the internet. XML Schema is one of several XML schema languages. Provides a set of rules to which an XML document must conform in order to be considered valid. An XML Schema instance is an XML Schema Definition (XSD). 15 Some Definitions - Continued JMS Elements: JMS Provider an implementation of the JMS interface for Message Oriented Middleware (MOM) JMS Client an application or process that produces and/or receives messages JMS Producer A JMS Client that creates and send messages JMS Consumer A JMS Client that receives messages JMS Message An object that contains the data being transferred between JMS clients. 16
Some Definitions - Continued JMS Elements - Continued: JMS Queue Staging area for message that have been sent and are waiting to be read. Messages are delivered in the order sent. A message is removed once it is read. JMS Topic A distribution mechanism for publishing messages that are delivered to multiple subscribers. 17 Some Definitions - Continued Differences and Similarities between JMS Topics and JMS Queues: Topics are used when the destination of a message is potentially more than one process Queues are used when the destination of a message is at most one process Topics and Queues are organized and named hierarchically Except for the use of a durable subscription, a process can only receive a copy of a message published to a topic if it is running and has an active subscription Message published to queues will remain on the queue until de-queued by a receiving process 18
Use Case Sequence Diagrams The document describes several use cases related to the interactions between components with a set of systems. Use Cases presented are from the perspective of the integration of systems; that is, the actors are defined in terms of the software systems that need to interact: (1) Client; (2) Server; (3) ESB, and; (4) Adapter Use Cases are described using Sequence Diagrams Three key terms related to messaging are request, reply and event and are reflected in terms of the verbs used to define the information flows. 19 Sequence Diagrams Request/Reply Client makes query request & server returns a set of objects Client makes a transaction request & server creates or modifies a set of objects Request uses get, create, update, delete, close or cancel as the verbs. 20
Sequence Diagrams Request/Reply w/esb Client processes can subscribe to listen for events Events use past tense verbs: (1) created; (2) updated; (3) deleted; (4) cancelled, and; (5) closed 21 Sequence Diagrams Events ESB decouples client from server Client does not have to know the location of the server or conform to the exact interface since the ESB completes the transformation Routing and mapping takes place in the integration layer 22
Sequence Diagrams Transactions Typically a combination of a request/reply for a message exchange with a consequential publication of events Requests related to transactions use the verbs: (1) create; (2) update; (3) delete; (4) cancel, and; (5) close 23 Sequence Diagrams Middleware Adapters When a System cannot directly connect to an ESB, an adapter is used to handle the connection. The system may be a database or a file directory. Adapter can generate events on behalf of the system 24
Sequence Diagrams Complex Messaging Used when a request involves many asynchronous replies to return information as it becomes available. Results are returned in the form of asynchronous events. These messages may take significant time to obtain results. 25 Sequence Diagrams Application Level Application-level messages are defined in the form of VERB(Noun) Verbs are Get, Reply, Create and Created Nouns are MeterReading and EndDeviceControls 26
Sequence Diagrams Application Level w/synchronous Replies For Synchronous Request/Reply message, the REPLY messages are assumed to be immediately returned. 27 Integration Patterns Four basic integration patterns are provided: Synchronous request/reply using Web Service Interface Synchronous request/reply using JMS Messages Asynchronous request/reply using JMS Messages Publish/Subscribe where the client is listening for JMS Messages referred to as events or notifications These patterns support the described Use Cases Allows more complex integration patterns using intermediaries within an ESB 28
Basic Web Service Pattern Client issues request to a Web Service Interface Interface is defined using a WSDL Client expects one of several results: Request is successfully processed and reply is returned in a timely manner Request is accepted but reply returns an application level error Request results in the return of a fault to the client No reply or fault is returned in a timely manner 29 Basic JMS Request/Reply Pattern Client sends a JMS Message to a Topic or Queue Reply may be synchronous or asynchronous The reply is optional Client expects one of several results: Request is successfully sent to a topic or queue and reply is returned in a timely manner Request is accepted but reply returns an application level error Attempt to send a Request to the topic or queue fails No reply is ever received 30
Event Listener Pattern A process listens for published events All subscribers to a topic will receive a copy of the message asynchronously JMS allows for durable subscriptions Event messages are send and consumed asynchronously 31 Custom Integration Patterns These would potentially include, but not be limited to patterns such as: Content-Based Router, where messages are routed based upon message content typically referenced using XPath expressions Smart Proxy, where messages may be redispatched to a specific destination service, where replies are accepted from the service and passed back to the client Claim Check, where a copy of an often very large file is maintained as a document for use by other processes, where the current status of the document is tracked, but the document is typically transported by means other than messaging Transformation, where transformations usually defined by XSL are used to reformat message contents Bridge, where a message published on a topic or queue may be forwarded to or received from another messaging infrastructure (this pattern can sometimes be implemented using third party products or simply through configuration) 32
Custom Integration Patterns Content-Based Router ESB Pattern using JMS allow Routing of Request Bus can make decisions related to the request handling 33 Custom Integration Patterns Smart Proxies Extends the Content-based router to permit a request to be initiated by a Web Service or a JMS Client Smart Proxies can dispatch the requests and correlate the responses 34
Custom Integration Patterns Service Integration Allows a service to expose its interface as a Web Service Adapter is in the ESB to convert the internal JMS message to the appropriate WS Request 35 Custom Integration Patterns Legacy Service Integration Have a Service that is not compliant to the standard Compliant Middleware Adapter is used in the ESB to provide a compliant interface 36
Message Flows All flows would typically use the ESB as an intermediary Events can be generated by servers or within the ESB Servers can also make client requests Some clients (and servers) will listen for events Transports used are web services and JMS 37 The Standards Based Integration Company Systems Integration Specialists Company, Inc. 61968 1-2: Web Services Implementation Profile CIM University San Francisco October 11, 2010 Margaret Goodrich, Manager, Systems Engineering SISCO, Inc. 6605 19½ Mile Road Sterling Heights, MI 48314 USA Tel: +1-903-477-7176 Fax: +1-903-489-0063 E-Mail: margaret@sisconet.com
Introduction Scope/Purpose Message Exchange Patterns Integration Patterns Other Topics: Message Organization Interface Specifications ESB Considerations Security Considerations 39 Scope/Purpose Provides recommendations for Web Service implementation using Service Oriented Architecture (SOA) service patterns and Web Services Description Language (WSDL). The Goal is to provide sufficient information to enable implementations to be interoperable. Will be used as an integral part of the interoperability tests for distribution management systems in 2010 and beyond. 40
Message Exchange Patterns Three basic types: One Way Two Way Call Back 41 Message Exchange Pattern: One Way The sender sends its request out but does not expect a response message back. Since typically HTTP protocol is used, the sender can still get HTTP-level information such as a HTTP 404 error for a failed server communication. sd One-way Sender Receiver Request() 42
Message Exchange Pattern: Two Way Is a synchronous process with typically two messages involved, one for request and one for response. A sender sends a request to a receiver who returns a response message back to the sender. The request message is indicated as Request and the response message Response. sd Two-way Sender Receiver Request() Response() 43 Message Exchange Pattern: Call Back The callback is an asynchronous process for message exchange. It is made of two request/response (initial and final) synchronous calls. The two are correlated in a way that each party can unambiguously identify which callback goes with which initial request. The Sender sends an initial request to Receiver with a message called InitialRequest. The Receiver receives the message, returns a response message back and invokes the final request/response call with a request message called FinalRequest. The whole call-back process is completed after the Sender replies the final request. 44
Message Exchange Pattern: Call Back sd Call-back Sender Receiver InitialRequest() InitialResponse() FinalRequest() FinalResponse() 45 Integration Patterns There are two Patterns Basic Patterns: Service Level Patterns these include Send, Request, Reply, Retrieve, Receive, Execute or Show Operation Level Patterns these include Create, Change, Cancel, Close, Delete, Created, Changed, Canceled, Closed, Deleted The Common Service/Operation Patterns are: Direct and Indirect Send-Receive Service Interaction 46
Integration Patterns - Continued Additional Common Service/Operation Patterns: Request-Execute-Reply-Receive Services Interaction Pattern Request-Execute Services Interaction Pattern Request-Receive Services Interaction Pattern Show-Receive Services Interaction Pattern Retrieve-Execute-Show-Receive Services Interaction Pattern Retrieve-Execute Services Interaction Pattern Execute-Reply Services Interaction Pattern 47 Integration Pattern: Direct & Indirect Two methods to complete a data transaction from one system/application to another: with an intermediary (e.g. ESB) - Indirect without an intermediary Direct Direct is also known as point-to-point integration. That is, only one service needs to be provided by a service provider. Indirect requires two services. 48
Integration Pattern: Direct sd Direct Direct Application A Application B Request() Reply() 49 Integration Pattern: Indirect sd Indirect Indirect Direct Application A Intermediary Application B Request() Reply() Request() Reply() 50
Integration Pattern: Send-Receive Services Interaction Application A sends its data through a Send Web service to an intermediary, such as an Enterprise Service Bus (ESB) Application B provides a Receive service in order to receive the data. Hence there are two Web services are involved. This is an indirect interaction process since Application A does not send its data directly to B but through ESB. It is an asynchronous process since multiple invocation threads are involved. 51 Integration Pattern: Send and Receive Services Interaction sd SendReceive Pattern Application A Direct Intermediary Application B CreatedWorkOrder() Acknowlegement() CreatedWorkOrder() Acknowledgement() 52
Integration Pattern: Request-Execute-Reply- Receive Services Interaction Application B sends a request to create, change, or cancel to Application A using one Request Web service and one Execute service provided by the ESB and Application A, respectively. After it retrieves the data, Application A calls a Reply service at the ESB which then passes the request onto Application B using a Receive service. Process happens in an asynchronous fashion, although each service call in this case is a synchronous call (acknowledgement as a reply). This is an indirect interaction process since Application B does not send a request directly to Application A, but through the ESB. It is an asynchronous process since multiple invocation threads are involved. There are four Web services involved in this case. 53 Integration Pattern: Request-Execute-Reply- Receive Services Interaction sd RequestExecuteReplyReceive Pattern Application A CreateWorkOrder() Direct Intermediary Application B Acknowlegement() CreateWorkOrder() Acknowlegement() CreatedWorkOrder() CreatedWorkOrder() Acknowlegement() Acknowledgement() 54
Integration Pattern: Request-Execute Services Interaction Application B sends a request to create, change, or cancel to Application A using one Request and/or one Execute Web services provided by ESB and Application A, respectively. This is a synchronous interaction pattern. There is no need to have a Reply or Receive service at Application B side to receive a reply from Application A since the original Request thread is still open. A Reply is simply an acknowledgement return from Application A in this case. This interaction may or may not go through ESB. If an ESB is in the middle, one Request and one Execute service are needed. If it is a Direct interaction between B and A, only an Execute service is needed. 55 Integration Pattern: Request-Execute Services Interaction sd RequestExecute Pattern Indirect Application A Direct Intermediary Application B CreateWorkOrder() CreateWorkOrder() Acknowledgement() Acknowledgement() 56
Integration Pattern: Request-Receive Services Interaction Application B requests a data from Application A using a Request service provided by Application A. After the data is retrieved, Application A sends the work order to Application B using a Reply service. This is similar to Request-Execute-Reply-Receive pattern in a sense that both are asynchronous interactions. However it does not include an ESB in the middle. It is a direct call from B to A for a request, and another call from A to B for reply. 57 Integration Pattern: Request-Receive Services Interaction sd Request/Reply Indirect Application B Application A CreateWorkOrder() Acknowledgement() CreatedWorkOrder() Acknowledgement() 58
Other Topics of IEC 61968-1-2 Additional patterns are being documented Message Organization Provides an overview of the message organization as defined in the Part 9 standard. Interface Specifications Gives a description of the WSDL Structure, document style SOAP binding, and MTOM (Message Transmission Optimization Mechanism) ESB Considerations Describes common ESB characteristics and recommendations and refers the reader to 61968-1-1. Security Considerations Discusses XML Encrytion and XML Signatures. 59 Discussion!!!! zzzzz 60
Questions & Contacts Margaret Goodrich Home Office: 903-489-1494 Cell: 903-477-7176 Email: margaret@sisconet.com 61