Towards Stream- based Reasoning and Machine Learning for IoT Applica<ons

Transcription

1 Towards Stream- based Reasoning and Machine Learning for IoT Applica<ons Markus Endler 1, Jean-Pierre Briot 2, Francisco Silva e Silva 3, Vitor P. Almeida 1, Edward H. Haeusler 1 1 PUC-Rio, Rio de Janeiro, Brazil 2 LIP6, Paris, France 3 UFMA, São Luiz, Brazil L A C Laboratory for Advanced Collaboration

2 Towards Real- <me Intelligence for IoT 2 General goals: Get a better understanding of what is happening in the physical world; Correlate apparently unrelated sensed events; Predict events that have not been sensed yet; Identify consequences of sensed events. Send out correct and timely actuation commands or notifications to humans.

3 Challenges of IoT Intelligence Sensors are multi-modal, distributed and heterogeneous Their data is noisy and incomplete Associated with time and space (Context) Availability and reliability are dynamic Must be processed fast several IoT applications require soft real-time analysis Privacy and security are important issues But data analysis alone is not sufficient! IoT demands ac7onable informa7on based on domain knowledge. 4

4 Outline The ESMOCYP Approach Example of Scenario Implemented Prototype Machine Learning Prospects Conclusion 6

5 Project ESMOCYP (1) : Merging three disciplines & Semantic Web & Reasoning Data Stream Processing (Complex Event Processing) Cyber-Physical Systems (IoT) (1) 7 Efficient Semantic MOdels and Fault-tolerant Middleware for CYber-Physical Systems

6 The ESMOCYP Approach Goal: achieve online reasoning as a combination of CEP and Deduction Logic reasoning Main steps: 1. Semantic annotation: Identify the type of entities (smart objects) and the action happening with it 2. Generate RDF streams (Semantic Event stream) from annotated sensor data 3. Use these Semantic Events to generate facts (Fact stream) using the Knowledge Base (KB), and also input new facts into the KB 4. In offline mode, process the KB and derive new Event Processing rules for online analysis 10 ESMOCYP relies on the ContextNet middleware [Endler et al, Middleware 2011], that has cloud and mobile components.

7 M-EPA: Mobile Event Processing Agent Mobile Hub [Rios/Endler:2015] is a middleware component for the Internet of Mobile Things (IoMT) M-EPA [Rios/Endler,2016] allows the Mobile Hub to pre-process local sensor data streams and send to the cloud just relevant information Performs Complex Event Processing (CEP) using ESPER engine Can identify relevant event patterns and generate complex events Can receive and deploy new CEP/EPL rules on the fly ContextNet services in the Cloud 11

8 Semantic Annotation of Simple Sensor Events/Data The M-EPA (Mobile Event Processing Agent) identifies the entity/ object and the action performed with the smart object The entity is identified by a mapping of UUID (Universal Unique Identifier) to ontology concepts The action is identified by the analysis of sensor data signal (e.g. value) UUID value UUID value Sensor data events UUID value Mobile EPA Action on entity: e.g. kicking (pr,sub) Predicate: moved Subject: cell phone UUID is cell phoney or UUID is temp. by sensorw UUID Mapping 12

9 From Data Event Stream to Semantic Event Stream Main Idea: An action (predicate) of a subject (sub) on an object (obj) occur at a unique instant of time and in a same place. Subject +Action Object +Action Annotated Sensor Data Annotated Sensor Event Stream (pr,sub) (pr,obj) (pr,obj) (pr,sub) Context mapper A Context mapper B Spatiotemporal correlations (sub,pr,obj) (sub,pr,obj) (sub,pr,obj) Semantic Event Stream Context-specific RDF triples 13

10 From Data Event Stream to Semantic Event Stream Annotated Data Event Stream (pr,sub) (pr,obj) (pr,obj) (pr,sub) Time window RDF +Timestamp + Conetxt Attributes Context mapper CEP rules link subjects and objects that have same/related predicates (actions), same timestamp and same location. This correlation is done by the M-EPA (close to the sensors) < (pr,sub), t, location > < (pr,obj), t, location > < (sub, pr, obj), t, context> 14

11 From Semantic Event Stream to Semantic Event Stream (RDF) Deduced Fact Stream Sub stream 1 (sub,pr,obj) (sub,pr,obj) (sub,pr,obj) Sub stream 2 (sub,pr,obj) (sub,pr,obj) (sub,pr,obj) Sub stream 3 (sub,pr,obj) (sub,pr,obj) (sub,pr,obj) Knowledge Base Semantic Event Rule A FACT A FACT B Semantic Event Rule B FACT C Knowledge facts Fact Stream 15

12 Example of Scenario Mr. Silva carries a smartphone running MobileHub and travels by bus with air conditioning and smart sensors MobileHub automatically discovers and connects to nearby smart sensors (with temperature and accelerometers) Analyzed data (from sensor location, accelerometer...): ambient condition (temperature), movement pattern... Deduced information: Silva is riding on Bus Line #435 Silva is in an air-conditioned ambient 16

13 Example of Scenario RDF events in the seman<c stream: (cell- phoney, connectedto, sensorx), (sensorx, reads, Abs(Accelerometer) > 10), (sensorx, reads, temp=20) Knowledge Base facts: (Silva, carries, cell- phoney) (sensorx,locatedin, BusKKZ8674), (BusKKZ8674, serves, BusLine 435). DL Rules: (cell- phone, connectedto, sensor) (sensor, in, ambient) => (cell- phone, in, ambient) (person, has, cell- phone) (cell- phone, in, ambient) => (person, in, ambient) (person, in, ambient) (ambient, isa, bus) (bus, servers, line) => (person, riding, line) (person, in, ambient) (sensor, in, ambient) (sensor, reads, data) => (person, experiences, data) 17

14 Prototype Level 1 uses Esper: receives the UUID of the Bluetooth sensor/beacon and the temperature values. By matching a sequence of data it will generate RDF triples with the entities type information. KB: Ontholgy has relations between entities (people, roles, devices, rooms, reachability, and time) Level 2: runs C-SPARQL (a continuous stream reasoning engine) that uses the KB. i.e., new facts can be deduced doing reasoning on their type hierarchy and their relationship to other objects in the application domain 18

15 Technologies used in the Prototype ESPER ESPER Event Processing Language (EPL) EPL: declarative (SQL-like) language with pattern language and sliding window Queries on Java objects and their attributes C-SPARQL Continuous/Stream Extension of SPARQL (SPARQL Protocol and RDF Query Language) Queries on stream of RDF triples Mobile Hub Android-based middleware for opportunisctically detecting, connecting and retrieving sensor data from Bluetooth Smart SensorTags M-EPA Android-based ESPER CEP engine where EPL rules can be downloaded and managed remotely (from the cloud) 19

16 Machine Learning Prospects Machine Learing is more appropriate for sta7s7cal analysis, and therefore should be employed in the early phases of the reasoning, e.g. to detect which phenomenon is occuring with the subjects/objects. Some ideas are: 1. Extract Features from Sensors Data Autoencoders, RBMs 2. Extract Paaerns from Features k- Means 3. Learn Temporal Series of Paaerns Recurrent Networks, Markov Chains Ex: SensorSax [Ganz et al. 2016]: 1; 2; 3 4. Deduc7on of abstract Facts and Rules Induc7ve Logic Programming 20

17 Conclusion Project ESMOCYP is exploring real- 7me reasoning for IoT systems, by combining Complex Event Processing and knowledge- based reasoning. So far, we are just exploring the technological feasibility Our prototype implementa7on has shown promising results 22 But many problems remain to be solved: How to handle facts that are only transient? What happens if the deduc7on of new facts is made on stale knowledge Knowledge base should have a no7on of 7me (for rela7ons and rules) How to deal with large ontologies? It is possible to clip/cut off the relevant part of the ontology? How to merge the modified clipped part into the whole ontology?

18 Thank you! Questions? Acknowledgement: CAPES-DAAD PROBRAL Cooperation Program For more information please concact: 23