OMF Documentation. Release OSIsoft, LLC

Transcription

1 OMF Documentation Release 0.11 OSIsoft, LLC December 24, 2016

2

3 Contents 1 Overview 1 2 Message Types Type Messages Object Messages Stream Messages Data Messages Headers Required Headers Optional Headers Samples 11 i

4 ii

5 CHAPTER 1 Overview The OSIsoft Message Format (OMF) defines a set of message headers and bodies that can be used to generate compliant messages for ingestion into an on-premises PI System or a cloud-hosted Qi archive. OMF can be used to develop data acquisition applications on platforms and in languages for which there are no supported OSIsoft libraries. OMF itself does not define or depend on any particular binary message protocol (HTTP, AMQP, Kafka, etc.) It is instead based on an abstract message type, where a message consists of a set of key / value pairs, called the headers, and a binary payload, called the body. OMF messages can thus be constructed using any message protocol that defines headers and bodies. For up to date information on the specific binary protocols currently supported by OSIsoft systems, consult the Qi and PI Connector Relay documentation. 1

6 OMF Documentation, Release Chapter 1. Overview

7 CHAPTER 2 Message Types Messages sent to a PI system fall into four categories: Type messages, Object messages, Stream messages, and Data messages. Within each category, three types of actions can be specified: Create, Update, and Delete. Each message can perform operations across multiple streams, types, or data objects by bulking each individual JSON object into a JSON array. To achieve optimal throughput, bulking and compression of messages is recommended. 2.1 Type Messages Each Stream, Span, Asset, and Link must be associated with a Type. Types are defined using the JSON Schema specification ( Any type used by a stream must be registered before the stream can be created. Types can be created, updated, or deleted. Updating a type without updating the type s version will result in failure. Once a type is deleted, any operations on streams or objects using that type will fail. The id keyword is required, and is used to associate a Stream, Span, Asset, or Link with a given data type. The classification keyword must be set to one of stream, asset, link, or span, and is used to denote whether the type is to be used to define Streams, Assets, Links, or Spans. If omitted, the type is assumed to be used to define Streams. An optional version keyword allows for multiple versions of the same type. If the version keyword is omitted, the version is assumed to be Once a type has been defined, the type s id cannot be reused unless a new version is specified. One property must be designated as the index by supplying the index keyword with a value of true. The designated index property is used to uniquely identify discrete Events, Spans, Assets and Links so that they can be updated or deleted after their initial creation. When declaring an Asset type, one property may be optionally designated as the name of an asset by supplying the name keyword with a value of true. If no property is declared to be the name, the property marked as the index will be used as the name. Because the index must be unique across all assets, the name keyword allows for multiple distinct assets to share a common name. Depending on the type s classification, additional properties are required: 3

8 OMF Documentation, Release 0.11 Type Classification Span Link Required Properties StartTime EndTime Source Target0... TargetN Sample Headers producertoken = b7cnvn36cq version = messagetype = type action = create Body [ { { "id": "TankMeasurement", "version": " ", "type": "object", "classification": "stream", "properties": { "Time": { "format": "date-time", "type": "string", "index": true "Pressure": { "type": "number" "Temperature": { "type": "number" "id": "Tank", "version": " ", "type": "object", "classification": "asset", "properties": { "Name": { "type": "string", "index": true "Serial": { "type": "string" "Model": { "type": "string" 4 Chapter 2. Message Types

9 OMF Documentation, Release 0.11 { { ] "id": "OverRange", "version": " ", "type": "object", "classification": "span", "properties": { "ID": { "type": "string", "index": true "StartTime": { "format": "date-time", "type": "string" "EndTime": { "format": "date-time", "type": "string" "id": "TankLink", "version": " ", "type": "object", "classification": "link", "properties": { "ID": { "type": "string", "index": true "Source": { "type": "string", "linktype": "Tank" "Target0": { "type": "string", "linktype": "Tank" "Target1": { "type": "string", "linktype": "Pump", "linktypeversion": " " 2.2 Object Messages Objects describe structural and relational data, as well as discrete events which are not associated with a continuous stream. Assets, Links, and Spans are all considered Objects Object Messages 5

10 OMF Documentation, Release Assets Assets represent metadata describing the device(s) being observed and should be used to capture data that is descriptive and relatively static in nature. In an on-premises PI system, Asset objects are mapped to AF elements. The cloud-hosted Qi system maps asset streams to Qi streams Links Links are used to define one-to-many relationships between Assets and other Assets, between Assets and Streams, and between Assets and Spans. Link types must define a single Source attribute, and N Target attributes, where the source attribute represents the index of an asset element, and the target attributes represent either the indices of other asset elements or span objects, or stream IDs of associated value streams. Each Source and Target attribute must include a linktype keyword which describes the type to be linked to or from, and optionally a linktypeversion keyword, which denotes the version of the linktype to use. On-premises PI systems interpret a link object by building child AF elements beneath the Source element for each target Asset attribute, Event Frames for each target Span attribute, and PI data references for each target Value stream attribute. AF elements can be built at the root level by sending a root link element with the Source attribute set to _ROOT and a Target attribute for each asset to be built at the root level. Child elements, Event Frames, and data references cannot be built until the parent asset has been built, either via a root link, or a chain of links that resolve up to a root link. Links are not presently supported in cloud-based systems Spans Spans are used to send data representing a time frame. Span types must contain StartTime and EndTime timestamp attributes. Optional descriptive attributes may accompany these attributes. On-premises PI systems interpret Span objects as Event Frames. Span data is not presently supported in Qi. Objects must conform to a pre-defined Stream, Asset, Span, or Link type. On-premises systems translate Asset objects to AF elements, and Span objects to Event Frames. Link objects generate relationships between AF elements, PI data references, and Event Frames. Once a type has been registered, object messages can be sent which use that type. Objects can be created, updated, or deleted. The body of an Object message consists of an array of objects made up of the following properties: Name type typeversion values Value ID of the type used by the object. Optional version of the type. If omitted, the highest version sent is assumed. An array of objects conforming to the type and typeversion. 6 Chapter 2. Message Types

11 OMF Documentation, Release Sample Headers producertoken = b7cnvn36cq version = messagetype = object action = create messageformat = json Body [ { ] "type": "Tank", "values": [ { "Name": "Building1", "Serial": " KKF4", "Model": "FN-2187" { "Name": "Building2", "Serial": " FS12", "Model": "TK-421" ] 2.3 Stream Messages A Stream is defined as a sequence of data events of identical type, where each event has a unique, definable index. Each Stream has a unique ID, defined upon creation of the Stream. Streams are designed to represent observed or calculated measurements taken from devices. Stream definition messages sent to an on-premises PI system must define an index property of timestamp type. Each additional property of the object is then mapped to a PI point. As Data messages associated with the Stream are sent to the system, they are timestamped using the index property. When sending Data messages to a cloud-hosted Qi system, the index attribute may be of any type. Each attribute defined in the stream type is collated into a single Qi stream, rather than split into multiple streams or points. Once a Type has been registered with a Type message, Streams can begin using that Type. Streams can be created, updated, or deleted. The body of a Stream message consists of an array of objects made up of the following properties: Name id type typeversion Value Unique identifier of the stream. ID of the type used by the stream. Optional version of the type used by the stream. If omitted, the highest version sent is assumed Sample Headers producertoken = b7cnvn36cq version = messagetype = stream action = create messageformat = json 2.3. Stream Messages 7

12 OMF Documentation, Release 0.11 Body [ {"id": "Building1TankMeasurements", "type": "TankMeasurement", "typeversion": " " {"id": "Building2TankMeasurements", "type": "TankMeasurement", "typeversion": " " ] 2.4 Data Messages Once usable Types and Streams have been created, Data messages can be posted to a Stream. Data messages can span multiple Streams. The body of a Data message is comprised of an array of key-value pairs, where the key is the Stream ID, and the value is an array of Data objects conforming to the given Stream s Type definition Sample Headers producertoken = b7cnvn36cq version = messagetype = data action = create messageformat = json Body [ { { ] "stream": "Building1TankMeasurements", "values": [ { "Time": " T22:33: Z", "Pressure": 25.4, "Temperature": { "Time": " T22:34: Z", "Pressure": 25.6, "Temperature": { "Time": " T22:35: Z", "Pressure": 25.7, "Temperature": { "Time": " T22:36: Z", "Pressure": 25.9, "Temperature": ] "stream": "Building2TankMeasurements", "values": [ { "Time": " T22:33: Z", "Pressure": 29.4, "Temperature": { "Time": " T22:34: Z", "Pressure": 30.1, "Temperature": { "Time": " T22:35: Z", "Pressure": 30.8, "Temperature": ] 8 Chapter 2. Message Types

13 CHAPTER 3 Headers The following headers are supported in version 0.11 of the message specification. Headers may be added to or removed from the specification over time. Backwards compatibility is achievable through the version header. The message processor uses the version header to determine whether a given message should be processed using older message specifications. 3.1 Required Headers producertoken A unique token generated by the destination system used to identify and authorize a given OMF producer. Consult the Qi or PI Connector Relay documentation for further information. messagetype Describes the type of message contained in the message body. One of: type, object, stream, or data. See Message Types. messageformat Describes the data serialization format employed in the message body. Currently limited to json. 3.2 Optional Headers version Specifies the version of the OSIsoft Message Format used in the message. Used by the message processor to determine how to process the remaining headers and body. If omitted, it is assumed to comply with the target system s default message version. If the message contains headers belonging to an older specification, the version should be included to ensure that the message will be processed appropriately. action One of: create, update, or delete. Describes the action to be performed using the data in the message body. If not specified, create is assumed. compression The compression algorithm used to compress the message body. Currently limited to gzip. If not specified, the message body is assumed to be uncompressed. 9

14 OMF Documentation, Release Chapter 3. Headers

15 CHAPTER 4 Samples Sample applications written in C, C#, Java, Node.js, and Python can be found at Samples 11