TC-IOT M2M CORE Services Protocol. User Manual. Version: 1.0 Date:

TC-IOT M2M CORE Services Protocol User Manual Version: 1.0 Date:

Document Name: TC-IOT M2M CORE Services Protocol - User Manual Version: 1.0 Date: Document ID: TC_IOT_M2M_CORE_Protocol_User_Manual_EN_v1.0 Content 1. Introduction... 3 2. What is the TC-IoT CORE?... 4 3. M2M CORE Services Protocol messages... 4 4. Methods... 6 5. Query - Searching for events... 9 6. Extra... 13 7. References... 14 2 of 14

1. Introduction This document provides an overview description of the M2M CORE Services Protocol, used in the Router component (TC-IoT CORE) of TC M2M Cloud Platform (TC-IoT) developed by Thought Creator. The objective of the document is to describe and illustrate the way to use the Protocol. For clarification, some terms used throughout the document are described in the following table. Term Description Client Router Gateway (Adapter) Terminal Subscription Feed User or Application receiving and sending information from/to the Terminal. Corresponds to the TC-IoT CORE component responsible for the queuing, distribution and archiving of events received from M2M Terminals. Software component responsible for the conversion of specific M2M communication protocols to the common and standard protocol used by the Router and by Clients. Electronic M2M Device able to interconnect with other equipments and/or sensors, and to communicate with the Gateway. Corresponds to the Registration of a Terminal in the system. Subscription made by a Client of the Terminal activates the automatic reception, in realtime, of all event issued by the said Terminal. Asynchronous Message originated from a Terminal sent to all Client who have Subscribed it. These messages correspond to state changes of Terminals or of their interconnected sensors. 3 of 14

2. What is the TC-IoT CORE? TC-IoT CORE is the main component of TC M2M Cloud Platform (TC-IoT), and is responsible for the distribution and storage of events from the M2M terminals. TC-IoT CORE is high-performance a message routing system, i.e., a Message Broker for all the traffic between Devices/Gateways and applications (Clients), controlling the authentication, authorization and access to Terminals, as well as replicating and storing messages of the subscribed Terminals. It can be considered a hybrid between a traditional router for distributing messages among resources, and a Resource Directory, a repository for registration and for querying information about those resources and their historical events. From a service perspective the TC-IoT CORE functions as a Web server, accessible from Web browsers and applications communicating via the HTTP protocol [1]. The TC-IoT CORE, i.e., the Router, interconnects the Clients (M2M applications), which consume information and request actions on Terminals, to the Gateways, which consist of applications (Adapters) that perform the translation of low-level device-specific M2M protocols, to the standard M2M CORE Services Protocol. Network Domain Gateway Domain Device Domain App App App App App M2M Application Services M2M CORE Services Protocol M2M Device Specific Messages GW Adapter M2M Device Access Networks GW Adapter M2M Device Resource Directory Application Layer API Layer IoT CORE Layer (Cloud) IP, 2G/GRPS, 3G/HSPA Agent Layer The M2M CORE Services Protocol is a resource-oriented and event-based communication protocol, over HTTP using a standard RESTful [2] design, or over WebSocket [3] transport, for real-time bi-directional communication of Terminal information. The protocol encodes requests and responses (messages exchanged between Gateway, Router and Client) using JSON [4,5], a textbased format of simple interpretation, widely supported by several programming languages and databases. This design makes it possible for the M2M CORE Services Protocol to be deployed at scale, by leveraging existing HTTP implementations and infrastructures. 3. M2M CORE Services Protocol messages The M2M CORE Services Protocol defines three distinct types of messages: information messages, terminal messages and control messages. The control messages are associated with quick response commands, such as subscriptions and registrations. The terminal messages are typically used for action commands or information requests between Client and Terminal and to query information about Terminals in the Resource Directory. 4 of 14

Control Messages A control message is used to inform the Client of the outcome of a particular operation, and consist of a JSON object with fields status and message. The message field indicates the result of the operation or the error that occurred, using an alphanumeric string format. The status field indicates if the operation was successful or not and typically takes one of the two values, 1 or -1. The status with a value of 1 indicates that the operation was successful. In case the value is -1, it means it was unable to execute the operation, as the following example: "message": "status": "NO_PERMISSION", - 1 Terminal Messages A terminal message can contain one or more JSON objects. There is no limitation on the content of these messages, only that they must contain the src, dst and timestamp fields. "src": "1234567", "dst": "1234567", "timestamp": 1388859768 The src field, for a message sent by a Client to a Terminal, must contain the username of the Client (the user with permissions to access the Terminal). For a message sent by a Terminal the src field must contain the identifier of the Terminal. The field dst defines the destination address of a message. In the case of a feed from a Terminal both fields src and dst have equal values corresponding to the identifier of the Terminal. The values of src and dst fileds are of alphanumeric string format. The timestamp field contains the creation date of the message represented in POSIX Epoch (Unix Timestamp), with an integer number format. Information Messages An information message is used to return Terminal identifiers known by the CORE. The responses have the format of JSON vector. 5 of 14

4. Methods Several methods (servlets) to interact with the TC-IoT CORE (Router component) are available. These methods allow the reception of messages, managing subscriptions, reading / changing states, etc. All requests should be made using the HTTP protocol. Optionally, for receiving and sending messages in real-time from / to the subscribed terminals, WebSockets [3] or HTTP Long Polling can be used. All requests require basic HTTP authentication using the credentials supplied by Thought Creator. The generality of the methods available in the Router function in a request-response perspective. The reception of messages related to information from subscribed Terminals, is sent to the Client whenever possible in real-time. If the Client is not online, those messages are temporarily stored in the TC-IoT platform and delivered a posteriori. Storage of messages for long periods are not guaranteed. It is advisable that Applications (Clients) are always connected to the TC-IoT Platform or to cancel subscriptions whenever these are no longer necessary. The next table describes the Methods and Resources supported by the Router, the format/syntax to be used and the entities can use them. Although available, it is not advisable to use the methods marked with (**) because they have lower performance and higher overhead in communications. Resource URI - Method - Message - request and/or Entity Description /index HTTP GET CONTROL MESSAGE All Returns the available information on the TC- IoT Platform. /subscribe_terminal/<id> HTTP GET CONTROL MESSAGE Client Subscribes the Terminal with the specified ID. Returns a control message with the result of the operation and, in case of error, the error code. /unsubscribe_terminal/ <ID> HTTP GET CONTROL MESSAGE Client Removes the subscription from Terminal with the specified ID. Returns a control message with the result of the operation and, in case of error, the error code. (**) /read_messages HTTP GET TERMINAL MESSAGE All Gets the oldest message stored in TC-IoT. If there are no saved messages, the HTTP session remains open for 30 seconds waiting for new messages. When the session expires a control message with timeout (Code 408) is received. 6 of 14

Resource URI - Method - Message - request and/or Entity Description (**) /message/<id> HTTP POST CONTROL MESSAGE request All Sends the message in the body of the POST request to the Client or Terminal with the specified ID. Returns a control message with the result of the operation. The ID parameter in the request is optional since the content of the message must contain the identifier of the recipient. /get_allowed HTTP GET INFORMATION MESSAGE [Vector of IDs] Client Returns the list of Terminals to which the Client can access, in a JSON vector format. /get_subscriptions HTTP GET INFORMATION MESSAGE [Vector of IDs] Client Returns the list of Terminals that the Client has subscribed, in a JSON vector format. /query HTTP POST TERMINAL MESSAGES request [Vector of messages] All Questions the TC-IoT through a query for information about a given set of terminals. (See example in Section 5) /messages WEBSOCKET Flow of TERMINAL MESSAGES All Bidirectional channel for sending messages between CORE and Client / Gateway. In direction Router -> Client, corresponds to messages from Terminals that Client has subscribed. In direction Client -> Terminal, corresponds to messages the Client wants to send to the Terminals. With this method, only terminal messages are allowed. (See section 3) /register_terminal/<id> HTTP POST CONTROL MESSAGE request Gateway This method allows to register a Terminal in the CORE. The POST body must contain the message for registering the Terminal. This method returns a control message. /unregister_terminal/<id> HTTP GET CONTROL MESSAGE Gateway This method allows removing the registration of a Terminal in the CORE. This method returns a control message. 7 of 14

Responses in Control Messages The following table describes the answers contained in the message field and the corresponding values of the status field for commands where the response is a control message: Message Status Description NO_PERMISSION -1 Message sent when there are no permissions to perform certain operations, such as subscribing to a Terminal not allowed. NOT_FOUND -1 Terminal not found. This message is received, for example, when a Terminal or a Client do not exist in the system. OK 1 When the operation was performed successfully and the message was sent. SUBSCRIBED, UNSUBSCRIBED, ALREADY SUBSCRIBED REGISTERED UNREGISTERED 1 When the operation was performed successfully for subscribing or unsubscribing Terminals. 1 When the operation was performed successfully for registration of Terminals. (Only for the account Gateway) BROADCAST_SENT 1 Feed message sent (Only for the account Gateway) MESSAGE_SENT 1 Message sent successfully to Client or Terminal. MESSAGE_ERROR -1 Error situation where it is not possible to send messages to a Terminal or to a Client. TIMEOUT_READING_MESSAGES 1 Message received on command /read_messages if there are no new messages at an interval of 30 seconds. Responses in Vector In the case of information messages, the responses return a JSON vector with the list of identifiers of Terminals, as shown in the following example: [123, 124, 1235] For terminal messages requests by HTTP POST, as is the case of /query command, the response returns a JSON vector with terminal messages. 8 of 14

HTTP Responses In parallel with the content of the control message it is possible from the HTTP response codes to get some information about the success of the operation, as shown in the following table. HTTP Responses Status Code and Reason Description 200 OK The action was successfully received, understood, and accepted. 403 Forbidden The request contains cannot be fulfilled. Typically, no permission to access the resource or no permission to access the Terminal. 404 Not Found Resource not available. 401 Authorization Required Authentication failure. 500 Internal Server Error This message may appear when the operation cannot be completed from the CORE side. 408 Request Timeout Timeout in /read_messages request. The response body is a control message where the field message has the value "TIMEOUT_READING_MESSAGES". 5. Query - Searching for events One of the features of the M2M Core Services Protocol is that all messages that are not control messages are stored persistently in the TC-IoT platform making possible at any time to query for them through the command /query. This section indicates the generic syntax used to query the fields and parameters contained in the messages stored in the TC-IoT platform, since the content of these messages differs depending on the type and model of the Terminal. The details of the fields and parameters of each Terminal model are described in relevant documents available on Thought Creator website. All searches are done using the servlet /query and are sent to the TC-IoT platform using an HTTP POST method, which content (request body) is a query in JSON format. In response, the TC-IoT platform returns a vector (array) of all messages matching the search performed. Response times differ depending on the complexity of the query and the amount of data involved. An example of a request /query is as follows: POST /query HTTP/1.1 Host: example.com Accept: application/json Content- Type: application/json Content- Length: 96 "src": "123456", "analog.value": $lt: 5 9 of 14

The example of the response to the request /query is as follows (returned an empty vector): HTTP/1.1 200 OK Content- Type: application/json Content- Length: 2 [] Response in Vector The TC-IoT always returns the set of messages that match the criteria presented. An example of response, which returns the search criterion relating to two analog ports of a Terminal, is the following: [ ], "analog": "port": 3, "value": 9.968519071116713, "src": "1234567", "dst": "1234567", "timestamp": 1389815505 "analog": "port": 3, "value": 9.967189654342079, "src": "1234567", "dst": "1234567", "timestamp": 1389816242 Non-specific Query Search requests on TC-IoT should be valid JSON objects containing all the conditions that must be present in the responses. Search requests in which the query is an empty JSON object (nonspecific), will return in response all the messages in the database: 10 of 14

Query conditions and filters Responses to a query can be filtered by both the "identifier" of the Terminal as any other set of identifying data fields. To search for messages that contain one or more criteria, they should be indicated in the content of the query. An example with only one search criterion, in which all messages are returned from the Terminal 123456, is as follows: "src": "123456" In another example, two fields are combined to limit the search. Such is the case of a search for messages containing analogue port 3 of Terminal 123456, where only the messages with information on that port are filtered: "src": "123456", "analog.port": 3 If it were omitted the src field all messages containing analog port 3, regardless of the Terminal in question were obtained. Some general conditions for search of messages are the following: $lt e $gt The conditions $gt (greater than), $lt (less than), allow to filter messages that contain an filed whose value is greater than or less than a certain numerical value. In the following example it is intended to retrieve all messages from Terminal 123456 with analog port values less than 5: "src": "123456", "analog.value": $lt: 5 11 of 14

Similarly for values greater than 9.94: "src": "123456", "analog.value": $gt: 9.94 $or The condition $or allows to make a search on messages satisfying at least one of the conditions imposed. In the following example the query combines the two filters exemplified above: "src": "1234567", $or: [ "analog.value": $gt: 9.94, "analog.value": $lt: 5 ] $ne The condition $ne (not-equal) allows searching for fields that do not have a certain value. "timestamp": $ne: 0 $exists The condition $exists allows to search for messages containing a given field. As an example, to obtain all messages containing the GPIO field, the query would be the following: "gpio": $exists: true 12 of 14

6. Extra In this chapter some additional information and suggestions for implementation are presented. Subscriptions For applications that are not constantly online it is advisable to unsubscribe the Terminals before the applications are placed offline. This prevents the accumulation of messages in TC-IoT platform while the applications are not connected. Receiving messages in real-time The reception of messages in real-time can be done through WebSockets or HTTP Long Polling. It is recommended preferably the WebSocket mechanism, in which case the Client just needs to keep one HTTP WebSocket session with TC-IoT platform. In case of using HTTP Long Polling it is necessary to pay attention to timeout.s It is advisable to always send a new request for each message successfully received. WebSockets The communication via WebSockets should respect the recommendations of RFC 6455 [3]. WebSockets support bidirectional, full-duplex communications over persistent connections. A WebSocket connection is established over a standard HTTP connection which is then upgraded without impacting the original connection, working with existing networking infrastructures including firewalls and proxies. To establish a WebSocket connection, the client side sends a WebSocket handshake request, for which the server side returns a WebSocket handshake response, as in the following example: GET /websock/proto/msg HTTP/1.1 Connection: Upgrade Upgrade: websocket Sec- WebSocket- Protocol: chat, better- chat Sec- WebSocket- Key: 50cLrugr7h3yAbe5Kpc52Q== Sec- WebSocket- Version: 13 Origin: http://example.com HTTP/1.1 101 Switching Protocols Connection: Upgrade Upgrade: WebSocket Sec- WebSocket- Accept: 58ij/Yod1NTjzqcyjkZbZk6V6v0= Sec- WebSocket- Protocol: chat Once the connection is established, the client and server can send WebSocket text frames back and forth in full-duplex mode. To keep the communications channel alive, it is necessary to send regular messages to indicate the channel is still being used. The PING/PONG WebSockets messages are designed to send non-application level traffic that will prevent the channel from being prematurely closed. A PING message may be sent by either side and is replied with a PONG message. The messages implemented in TC-IoT are in text format. 13 of 14

7. References [1] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee, Hypertext Transfer Protocol HTTP/1.1, RFC 2616, Internet Engineering Task Force, Jun. 1999. Updated by RFC 2817. [2] R. T. Fielding, REST: Architectural Styles and the Design of Network-based Software Architectures. Doctoral dissertation, University of California, Irvine, 2000. [3] I. Fette and A. Melnikov, The WebSocket Protocol, RFC 6455, Internet Engineering Task Force, Dec. 2011. [4] D. Crockford, The application/json Media Type for JavaScript Object Notation (JSON), RFC 4627, Internet Engineering Task Force, Jul. 2006. Updated by RFC 7158, 7159. [5] T. Bray (Ed.), The JavaScript Object Notation (JSON) Data Interchange Format, RFC 7159, Internet Engineering Task Force, Mar. 2014. 14 of 14