RESTful CIM using the Open Data Protocol Alan McMorran B.Eng Ph.D!1
CIM Exchanges!2
CIM Data Exchange The exchange of data using CIM has traditionally been used in two primary ways: Full Model exchange as CIM RDF XML Message based exchange as CIM XSD CIM RDF XML and CIM XSD cover two common use-cases: CIM RDF XML supports Network Model Management (NMM) exchanges both within and between companies and organisations CIM XSD exchanges support intra-system exchanges using an Enterprise Service Bus and/or web services!3
Hierarchical vs Graph!4
Hierarchical vs Graph RDF is very well suited to non-directed graph data XSD is better suited for hierarchical data Electrical network data rarely fits neatly into a hierarchy!5
CIM XSD Standards The IEC 61968 series of standards is primarily focussed on defining CIM XSD based messages These messages are explicitly defined hierarchies that can be used by Web Services Examples include: IEC 61968-3 Network Operations IEC 61968-6 Maintenance & Construction IEC 61968-9 Meter Reading and Control!6
Message Payloads These standards define fixed message payloads of hierarchical CIM data These payloads are defined within the standard and implemented by vendors They are combined with standard verbs (CREATE, UPDATE, DELETE etc.)!7
Services This means that every payload must be defined and agreed between all parties This approach has scalability issues As more parties connect; more services are interconnected; and more CIM data is being shared a more flexible approach is needed Systems should be capable of sharing CIM data without requiring a full system export or all parties to agree a fixed subset and hierarchy for every possible service!8
RESTful Services!9
What is RESTful? Representational State Transfer (REST) is an architectural style that defines a set of constraints and properties based on HyperText Transfer Protocol (HTTP)" REST-compliant web services allow the requesting systems to access and manipulate textual representations of web resources by using a uniform and predefined set of stateless operations Wikipedia!10
RESTful Principles All resources are uniquely addressable, usually through Uniform Resource Identifiers (URI) All resources can be manipulated through a constrained set of well-known actions, usually CRUD (Create, Read, Update, Delete) The data is transferred through any of a constrained number of well-known representations (e.g. HTML, XML, JSON) The communications between client and application are stateless allowing for multiple layers of intermediaries, caching etc.!11
RESTFUL Example Read (HTTP GET) HTTP GET https://opengrid.com/myservice/001 <No Content> Payload { } Result "type":"example", "name":"get", "id": "001", "company":"", "web":"https://"!12
RESTful Example Create (HTTP POST) HTTP POST https://opengrid.com/myservice { } Payload "type":"example", "name":"post", "company":"statnett", web":"http://www.statnett.no" { } Result "id": "002",!13
RESTful Example Update (HTTP POST) HTTP POST https://opengrid.com/myservice/002 { } Payload web : http://www.statnett.no/en/" Status: HTTP 204 No Content Result!14
RESTful Example Delete (HTTP DELETE) HTTP DELETE https://opengrid.com/myservice/002 <No Content> Payload Status: HTTP 204 No Content Result!15
RESTful Benefits All communications go through a single uniform interface With HTTP protocols and JSON/XML payloads the services can be accessed by any modern language Simple, easy to use architecture Web Browsers use RESTful services over HTTP Standard verbs, response codes, data types etc. Flexible service endpoint structures!16
Challenges This flexibility allows for different but valid URI patterns: https://opengrid.com/myservice/002 https://opengrid.com/myservice?id=002 https://opengrid.com/myservice(id='002') All valid URI structures The payload formats are commonly JSON or XML but there is no restrictions The contents of the payloads can follow any structure For interoperability, these must be restricted and standardised!17
OData!18
OData Background The Open Data Protocol (OData) is a protocol standard that defines a set of best practices for building and consuming RESTful APIs OData was initiated by Microsoft in 2007 and since version 4.0 has been under the OASIS OData Technical Committee This was submitted to the ISO/IEC as an international standard Allows the creation and consumption of REST APIs using meta-data driven URLs and and standard HTTP protocols and messages!19
OData Enhancements Standard URI structure for RESTful service endpoints Standard Query format Supports client-selectable JSON or XML formats for payloads/results Supported by existing BI and office applications (including Excel and Sharepoint) OData libraries for.net, Java, Javascript, PHP, C++ (but can be access as plain HTTP RESTful service without a library in any language that supports HTTP) Fully meta-data driven with an Entity Data Model (EDM)!20
Entity Data Model Standard UML components map one-to-one to the Entity Data Model UML EDM Package Schema Class EntityType SuperClass BaseClass Attribute Property Association NavigationProperty Full support for data typing (e.g. String, Boolean, Floating Point, Integer) that map to XSD types!21
RESTful with MetaData This provides a fully meta-model driven RESTful endpoint for managing data The CIM can be fully mapped into the EDM including all inheritance, associations, attributes and data types!22
Extended Payloads Synchronous Machine as plain JSON { } "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"turner V1 SM", "p":0.0, "q":0.8, "maxq":50.0, "minq":-17.0, "operatingmode":1, "type":1, Synchronous Machine as JSON with extended meta-data { "@odata.context":"http://localhost:8080/cimphony/concert/db/ resources.svc$metadata#cimphony.concert.cim16a.iec61970.base.wires.synchronousmachine/$entity", "@odata.type":"#cimphony.concert.cim16a.iec61970.base.wires.synchronousmachine", "@odata.id":"concertresource(id='6f4c8710-6995-11e8-b5cf-96b18064619b')/contents/cimphony.concert.resource.cim16a/ SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "@odata.editlink":"concertresource(id='6f4c8710-6995-11e8-b5cf-96b18064619b')/contents/cimphony.concert.resource.cim16a/ SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"turner V1 SM", "p":0.0, "q":0.8, "maxq":50.0, "minq":-17.0, "operatingmode@odata.type":"#int32", "operatingmode":1, "type@odata.type":"#int32", "type":1, }!23
Meta-Data Defined Services The EDM is used by OData aware clients and servers to validate communications Clients can request the EDM from the endpoint in a standard format to perform auto-discovery and clientside validation Servers will validate all endpoint URIs and service payloads against the EDM This allows for meta-data driven validation of data at the interface before it reaches the application itself!24
OData Example OData abstracts the interface from a particular format so can provide XML or JSON representations Endpoints can also be configured to returning other formats e.g. SVG, PNG or RDF XML through an OData interface The standard RESTful URL structure is all based on the meta-model: http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ Contents/Cimphony.Concert.Resource.CIM15/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')!25
{ "@odata.context":"http://localhost:8080/cimphony/concert/db/management.svc$metadata#concertresource", "value":[ http(s)://<host>/cimphony/concert/db/management.svc/ { ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ "@odata.mediacontenttype":"application/rdf+xml", "@odata.type":"#cimphony.concert.versioncontrol.management.concertresource", Contents/Cimphony.Concert.Resource.CIM15/ "@odata.id":"concertresource(id='de6931d0-4212-11e7-b565-96b6cd723f39')", "ID":"de6931d0-4212-11e7-b565-96b6cd723f39", SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725') "head":"0c1dc960-4213-11e7-b565-96b6cd723f39", "name":"alstom 60", "description":null, "updated@odata.type":"#datetimeoffset", "updated":"2017-05-26t12:58:43.318z", "created@odata.type":"#datetimeoffset", "created":"2017-05-26t12:57:26.637z", "path@odata.type":"#collection(string)", "path":[ "EMS Examples" ], "frameid":null, "checkedout":null, "checkedoutby":null, "owner":"mcmorran", "version":null, "schemas@odata.type":"#collection(string)", "schemas":[ "http://iec.ch/tc57/2010/cim-schema-cim15" ], "Branches@odata.navigationLink":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')/Branches", "Contents@odata.navigationLink":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15" }, { "@odata.mediacontenttype":"application/rdf+xml", "@odata.type":"#cimphony.concert.versioncontrol.management.concertresource", "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')", "ID":"d8815f90-41db-11e7-b565-96b6cd723f39", "head":"d9d2bec0-41db-11e7-b565-96b6cd723f39", "name":"abb 40", "description":null, "updated@odata.type":"#datetimeoffset",!26
{ http(s)://<host>/cimphony/concert/db/management.svc/ "@odata.context":"$metadata#cimphony.concert.resource.cim15/$entity", "@odata.type":"#cimphony.concert.resource.cim15", ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/cimphony.concert.resource.cim15", Contents/Cimphony.Concert.Resource.CIM15/ "ID":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15", "GeographicalRegion@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/GeographicalRegion", SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725') "DayType@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/DayType", "BaseVoltage@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/BaseVoltage", "RegularTimePoint@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/RegularTimePoint", "PowerTransformer@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/PowerTransformer", "Analog@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/Analog", "ConformLoadSchedule@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/ConformLoadSchedule", "ApparentPowerLimit@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/ApparentPowerLimit", "ShuntCompensator@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/ShuntCompensator", "Unit@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/Unit", "RatioTapChanger@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/RatioTapChanger", "NonConformLoad@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/NonConformLoad", "PowerTransformerEnd@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/PowerTransformerEnd", "VoltageLimit@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/VoltageLimit", "OperationalLimitSet@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/OperationalLimitSet", "VoltageLevel@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/VoltageLevel", "RegulationSchedule@odata.navigationLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/RegulationSchedule",!27
{ "@odata.context":"http://localhost:8080/cimphony/concert/db/ management.svc$metadata#concertresource", "value":[ { Contents/Cimphony.Concert.Resource.CIM15/ "@odata.type":"#cimphony.concert.cim15.iec61970.core.subgeographicalregion", "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "@odata.editlink":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "UUID":"_9497C00D7BFE44D59A57C65D8B76B725", "mrid":null, "aliasname":null, "name":"abb40apc" }, { "@odata.type":"#cimphony.concert.cim15.iec61970.core.subgeographicalregion", "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_B7CD48062F2A49358F35F54AEB2D4008')", "@odata.editlink":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_B7CD48062F2A49358F35F54AEB2D4008')", "UUID":"_B7CD48062F2A49358F35F54AEB2D4008", "mrid":null, "aliasname":null, "name":"abb40north" }, { "@odata.type":"#cimphony.concert.cim15.iec61970.core.subgeographicalregion", "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_04D6363BC30E4663B7AAF08AC930A9A9')", "@odata.editlink":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_04D6363BC30E4663B7AAF08AC930A9A9')", "UUID":"_04D6363BC30E4663B7AAF08AC930A9A9", "mrid":null, "aliasname":null, "name":"abb40south" }, { "@odata.type":"#cimphony.concert.cim15.iec61970.core.subgeographicalregion", http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')!28
{ "@odata.context":"$metadata#cimphony.concert.cim15.iec61970.core.subgeographicalregion/$entity", "@odata.type":"#cimphony.concert.cim15.iec61970.core.subgeographicalregion", http(s)://<host>/cimphony/concert/db/management.svc/ "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "@odata.editlink":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", Contents/Cimphony.Concert.Resource.CIM15/ "UUID":"_9497C00D7BFE44D59A57C65D8B76B725", "mrid":null, "aliasname":null, "name":"abb40apc", "Names":[ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725') ], "DiagramObjects":[ ], "ModelingAuthoritySet":null, "Substations":[ { "@odata.type":"#cimphony.concert.cim15.iec61970.core.substation", "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "@odata.editlink":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "UUID":"_58F637D8B03A4B12A67DF2E5797F9B6A", "mrid":null, "aliasname":null, "name":"crossplains" }, { "@odata.type":"#cimphony.concert.cim15.iec61970.core.substation", "@odata.id":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "@odata.editlink":"concertresource(id='d8815f90-41db-11e7-b565-96b6cd723f39')/contents/ Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "UUID":"_7324D6723635494784A4D8A9578FCE8A", "mrid":null,!29
OData in Action for DERMS!30
DERMS Project is providing the CIM data management repository for an ongoing DERMS innovation project at a large UK distribution company Multiple internal and external systems all provide and consume CIM data (including real-time data)!31
CIM Data The distribution network model from the DMS Geographical data integrated with the DMS model DER market data including availability/pricing TSO market queries and requests Outputs from market/system simulation engines Real-time SCADA data!31
Expanding Interfaces The number of requested interfaces is constantly growing Requests for access to different streams of data based on application requirements The CIM repository provides an OData interface to the static as-built network data and the real-time data Role Based Access Control to allow different users to view/update different sets of data OData allows most data access requests to be met with the generic service!32
Example Queries We need a list of the current real and reactive power set points for all the DER on the network The CIM class is PowerElectronicsConnection Only need the p and q attributes values for each http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID= 6f4c8710-6995-11e8-b5cf-96b18064619b )/ Contents/Cimphony.Concert.Resource.CIM16A/ PowerElectronicsConnection? $select=p,q!33
Example Queries Oh but can we only get DER that are solar? We need to check the PowerElectronicsUnit association of the device is of type PhotoVoltaicUnit http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID= 6f4c8710-6995-11e8-b5cf-96b18064619b )/ Contents/Cimphony.Concert.Resource.CIM16A/ PowerElectronicsConnection? $select=p,q& $filter=isof( PowerElectronicsUnit,Cimphony.Concert.CIM16A.E xtension.generation.production.photovoltaicunit)!34
Example Queries We need an endpoint to return all the SCADA measurement points attached to circuit breakers that are monitoring the switch status The CIM class is Discrete (subclass of Measurement) Need all instances that associate with a Breaker via the PowerSystemResource association and have a measurementtype of SwitchStatus http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID= 6f4c8710-6995-11e8-b5cf-96b18064619b )/ Contents/Cimphony.Concert.Resource.CIM16A/Discrete? $filter=isof( PowerSystemResource,Cimphony.Concert.CIM16A.IE C61970.Base.Wires.Breaker) and measurementtype eq SwitchStatus!35
Services as Queries The same approach is used for updates Applications can run a CIM query for specific CIM object(s) then use the returned data to run subsequent queries to update the data (or run more queries) e.g. Updating a SCADA measurement involves querying for the CIM Analog with the desire ICCP name The query includes an instruction to also include the associated AnalogValue ($expand=analogvalue) A subsequent update changes the value of the AnalogValue with the returned ID!36
Conclusion!37
Summary Increasing numbers of systems, applications, and services want to access and update CIM data A generic, platform-independent, metadata-driven service provides flexibility for users to use CIM data OData solves many of the problems by defining standard best practices and libraries for RESTful HTTP services Existing support in common applications, programming languages and platforms Can still be used as a light-weight plain RESTful service for clients that are not OData-aware A single endpoint to support multiple services!38
Questions alan@opengrid.com!39