Customization and Provisioning of Complex Event Processing using TOSCA IAAS & IPVS Ana C. Franco da Silva, Pascal Hirmer, Uwe Breitenbücher, Oliver Kopp, Bernhard Mitschang franco-da-silva@informatik.uni-stuttgart.de
Motivation Internet of Things Pervasive presence of smart devices, equipped with sensors, and exchanging information New IoT applications such as smart homes, smart factories, smart cities Important issues: timely data processing amount of exchanged data among devices established approach: Complex Event Processing (CEP) 2
Motivation Complex Event Processing CEP systems find event patterns based on data streams filter and aggregate data Many heterogeneous CEP systems exist Set up of customized CEP systems requires configuration of the system binding of data sources and data sinks deployment of CEP queries manual setup is cumbersome and time-consuming 3
Contribution & Outline Contribution: Automated provisioning of customized CEP systems based on the TOSCA standard Outline: TOSCA Overview Customization and Provisioning of CEP using TOSCA Case Study Summary and Future Work 4
TOSCA Overview 5
TOSCA Overview Topology and Orchestration Specification for Cloud Applications CEP System Web Server Provision VM & OS Install Web Server Deploy CEP System Operating System Service Orchestration for Deployment and Management Cloud Provider VM Application Topology 6
Graphical Example of an Application Topology CEP System (War) Node Template Operations Properties install deploywar Web Server (Tomcat) User: Admin PW: ****.sh.zip Implementation Artifacts (IA) Deployment Artifacts (DA) (hostedon) Relationship Template Operating System (Ubuntu14.04VM) Cloud Provider VM (OpenStack) 7
Customization and Provisioning of CEP using TOSCA 8
Main Contribution Customization and Provisioning of CEP Configuration of CEP system Binding of data sources and sinks Deployment of CEP queries Modeling of the TOSCA topology Deployment into a TOSCA runtime environment Instantiation of the CEP system setup 9
Topology for customized CEP systems Source adapter CEP Query Sink adapter T1 (Topic) CEP Source Wrapper CEP Customization (CEP-Customization) CEP Sink Wrapper T3 (Topic) CEP System T2 (Topic) System-Runtime (Runtime) connectsto hostedon dependson Node Template Deployment Artifact data flow 10
Data Processing Stack Data producer stack Data processing stack Data consumer stack Source adapter CEP Query Sink adapter T1 (Topic) CEP Source Wrapper CEP Customization (CEPCustomization) CEP Sink Wrapper T3 (Topic) Message Broker-1 (MessageBroker) CEP System Message Broker-2 (MessageBroker) Broker-OS-1 (OperatingSystem) System-Runtime (Runtime) Broker-OS-2 (OperatingSystem) Hypervisor-1 (Hypervisor) CEP-System-OS (OperatingSystem) Hypervisor-2 (Hypervisor) Hypervisor-3 (Hypervisor) connectsto hostedon dependson Node Template Deployment Artifact data flow 11
Data Producer Stack Data producer stack Data processing stack Source adapter CEP Query Adapter T1 (Topic) CEP Source Wrapper CEP Customizatio (CEPCustomizatio RaspberryPi-1-OS (OperatingSystem) Message Broker-1 (MessageBroker) CEP System RaspberryPi-1 (RaspberryPi) Broker-OS-1 (OperatingSystem) System-Runtime (Runtime) Hypervisor-1 (Hypervisor) CEP-System-OS (OperatingSystem Hypervisor-3 (Hypervisor) connectsto hostedon dependson Node Template Deployment Artifact data flow 12
Data Consumer Stack Data processing stack Data consumer stack CEP Query Sink adapter Customization PCustomization) CEP Sink Wrapper T3 (Topic) Consumer App CEP System Message Broker-2 (MessageBroker) RaspberryPi-2-OS (OperatingSystem) stem-runtime (Runtime) Broker-OS-2 (OperatingSystem) RaspberryPi-2 (RaspberryPi) EP-System-OS eratingsystem) Hypervisor-2 (Hypervisor) Hypervisor-3 (Hypervisor) connectsto hostedon dependson Node Template Deployment Artifact data flow 13
Case Study 14
Temperature Monitoring Data producer: Raspberry Pi measures the temperature Temperature values are provided to the CEP system Data processing: CEP system processes values and creates output events Turn the ventilator ON or OFF Data consumer: Raspberry Pi subscribes to topic command Actuator triggered to turn the ventilator ON or OFF 15
Experimental Setup Push events CEP System Environment Esper engine Push events Source adapter Sink adapter Subscribe to Topic T1 Publish to Topic T3 Raspberry Pi Adapter Publish value in Topic T1 T1 Message Broker T3 Subscribe to Topic T3 Raspberry Pi Consumer App Read Value ADC Send ON/OFF USB Temperature sensor GPIO 433 MHz Ventilator 16
Experimental Setup 17
Topology Model Data producer stack Data processing stack Data consumer stack Source adapter CEP Query Sink adapter Adapter (PythonApp) T1 (Topic) CEP Source Wrapper CEP Customization (CEPCustomization) CEP Sink Wrapper T3 (Topic) Consumer App (PythonApp) RaspberryPi-1-OS (RaspbianJessie) MessageBroker-1 (Mosquitto_3.1) Esper engine MessageBroker-2 (Mosquitto_3.1) RaspberryPi-2-OS (RaspbianJessie) RaspberryPi-1 (RaspberryPi3) Broker-OS-1 (Ubuntu14.04VM) Tomcat (Tomcat) Broker-OS-2 (Ubuntu14.04VM) RaspberryPi-2 (RaspberryPi3) Hypervisor-1 (OpenStack) CEP-System-OS (Ubuntu14.04VM) Hypervisor-2 (OpenStack) Hypervisor-3 (OpenStack) connectsto hostedon dependson Node Template Deployment Artifact data flow 18
Summary and Future Work Our approach enables Automated setup of customized CEP systems Easy exchangeability of CEP systems Future work Creation of more TOSCA Node Types to enable a wider range of CEP systems Distribution of CEP using TOSCA 19
Customization and Provisioning of Complex Event Processing using TOSCA IAAS & IPVS Ana C. Franco da Silva, Pascal Hirmer, Uwe Breitenbücher, Oliver Kopp, Bernhard Mitschang franco-da-silva@informatik.uni-stuttgart.de