SitRS XT Towards Near Real Time Situation Recognition

Transcription

1 SitRS XT Towards Near Real Time Recognition Ana Cristina Franco da Silva, Pascal Hirmer, Matthias Wieland, Bernhard Mitschang Institute of Parallel and Distributed Systems, University of Stuttgart, Stuttgart, Germany

2 Motivation Internet of Things Pervasive presence of smart devices, equipped with sensors, and exchanging information Huge amounts of sensors generating data data enables the recognition of events and a proper reaction to them Uniform sensor data integration and sensor data processing needed Ana Cristina Franco da Silva 2

3 Motivation Recognition : A change of state in a smart environment (e.g., smart factory, smart home) s recognized based on sensor data But: huge amount of low-level, raw sensor data Near real time is an important requirement, especially in manufacturing environments We need a means to efficiently derive high-level situations based on raw sensor data Ana Cristina Franco da Silva 3

4 Paper Contribution and Agenda Architecture and a stepwise approach for situation recognition SitRS XT Easy modeling of situations Process raw sensor data using Complex Event Processing to recognize situations in near real time Agenda (1) modeling with Templates (2) SitRS XT stepwise approach (3) SitRS XT architecture (4) Demonstration (5) Summary and outlook Ana Cristina Franco da Silva 4

5 Recognition Pyramid Levels Application Level: -aware Application 1. Register situation recognition for machine 2. Adapt to situations Knowledge Level: -Model, e.g. machine is overheated Information Level: Observable Context Context-Model, e.g. Machine.Temp Data Level: Data observation notification Basic data types, e.g. C Smart Environment Level: Observable Objects (Devices) Ana Cristina Franco da Silva 5

6 Example of Template : Production machine is overheated Machine overheated High-Level OR Logical Operation > 90 > 90 Conditions Temp. in C Temp. in C s Temp. sensor 1 Temp. sensor 2 Ana Cristina Franco da Silva 6

7 SitRS XT Stepwise Approach XML Schema Definition <xs:schema> <xs:element> </xs:element> </xs:schema> Template CEP Queries Select * From Pattern [ ] Flows Registration Modeling Registration Transformation Recognition Parse XML Deploy Queries Technical expert expert Ana Cristina Franco da Silva 7

8 Use case Monitoring objects on a conveyor belt Goal: detecting wrongly positioned objects on a conveyor belt Top Distance Left Distance Right Distance 1 Object detected 3 Outside limits 2 Upside-down Ana Cristina Franco da Silva 8

9 Modeling 1 Registration Modeling Registration Transformation 5 Recognition Wrongly positioned Sub-situation 1: Object detected AND Sub-situation 2: Upside-down OR Sub-situation 3: Outside limits time: 500ms < < x < 60 in cm in cm in cm in cm Top Distance Top Distance Left Distance Right Distance Ana Cristina Franco da Silva 9

10 Transformation 1 Registration Modeling Registration Transformation 5 Recognition Templates (ST) in XML mapped onto Event Processing Language Set of conditions corresponds to an event pattern situation node operation node select * from pattern [ <conditionnode_patternexpression> <operationnode_type> <conditionnode_patternexpression> ] condition node context node condition node context node conditionnode_patternexpression = <sensor_type_stream> ( sensor_id = <monitored_sensor_id>, <conditionnode_condition>) Ana Cristina Franco da Silva 10

11 Transformation Time-based conditions Object detected on conveyor belt: Distance from top distance sensor is lower than the distance between sensor and conveyor belt (i.e., 60 cm) for 500 milliseconds Detected < 60 in cm Top Distance for 500 ms select * from pattern [ every A0_stream=DistanceStream( sensor_id= Top Distance, distance < 60) -> (timer:interval (500 msec) and not DistanceStream( sensor_id= Top Distance, distance >= 60)) ] Ana Cristina Franco da Silva 11

12 Transformation value comparison Outside limits: Object is not centralized, i.e., the distance from the left distance sensor and the right distance sensor is not equal. in cm Outside limits!= in cm select * from pattern [ every A0_stream=DistanceStream( sensor_id = Left Distance ) -> A1_stream=DistanceStream( sensor_id = Right Distance, A0_stream.distance!= distance) ] Left Distance Right Distance Ana Cristina Franco da Silva 12

13 Transformation of a complex situation time: 500ms < 60 in cm Top Distance Sub-situation 1: Object detected Sub-situation 2: Upside-down 50 < x < 60 in cm Top Distance Wrongly positioned AND OR in cm Left Distance Sub-situation 3: Outside limits!= in cm Right Distance select * from pattern [ (every A2_stream = DistanceStream(sensorID = 'disttop', distance < 60) -> (timer:interval (500msec) and not DistanceStream(sensorID = 'disttop', distance > 60))) and (( every A1_stream = DistanceStream(sensorID='dist Top', distance > 50, distance < 60)) or ( every A0_stream = DistanceStream(sensorID = 'distleft') -> (A1_stream = DistanceStream(sensorID = 'distright', 0_stream.distance!= distance) and not distancestream(sensorid = 'distleft')))) ] Ana Cristina Franco da Silva 13

14 Step 5: Recognition Result of Recognition Object wrongly positioned object Observed object: Conveyor belt Location: , Timestamp: Description: Object is positioned wrongly and has to be reallocated. ST-Processing: ST632 wrongly positioned Recognition-System: CEP Engine Ana Cristina Franco da Silva 14

15 SitRS XT Architecture 1 Registration Modeling Registration Transformation 5 Recognition -aware applications select template 3 register situation recognition notify Registration Service and Context Model Template Repository load Template to CEP query Transformer trigger transformation deploy 4 5 publish situation occurrence model template 2 Complex Event Processing engine Expert push sensor data Recognition pull sensor data 1 Recognition Service Layer Message Broker Resource Management Platform data cache bind Registry Physical things with sensors

16 SitRS XT Prototype and Measurements Template to CEP query Transformer Notify situation-aware applications deploy MQTT Client Esper CEP engine push sensor data MQTT Broker Resource Management Platform Recognition Service Publish sensor data to topic distance MQTT Client Data Reader pull Raspberry Pi GPIO Distance sensors # situations in parallel runtime /situation Node-RED CEP Esper ms 3 ms ms 6.96 ms ms ms ms ms ms Ana Cristina Franco da Silva 16

17 Demonstration Recognizes when object is outside limits SitRS XT MQTT MQTT Ana Cristina Franco da Silva 17

18 Demonstration Ana Cristina Franco da Silva 18

19 Summary and Outlook SitRS XT is a general purpose, situation recognition approach that can be used in different use-cases Summary Initial goal: Recognize situations in near real time based on raw sensor data achieved Templates to model situations Complex Event Processing based situation recognition in near real time Future work To guarantee predefined recognition times through scalability and availability Ana Cristina Franco da Silva 19

20 SitRS XT Towards Near Real Time Recognition Ana Cristina Franco da Silva, Pascal Hirmer, Matthias Wieland, Bernhard Mitschang Institute of Parallel and Distributed Systems, University of Stuttgart, Stuttgart, Germany