Bridge IT and OT with a process data warehouse Presented by Franco Camba, OSIsoft Matt Ziegler, OSIsoft Frank Ruland, SAP
Audience Poll Have you invested or are you looking into Business Intelligence tools? How many of you are running or considering a Big Data Project? Of these tools/technologies, which are the ones you are looking to integrate Operational data into? a) PowerBI b) Qlik c) Tableau d) Tibco Spotfire e) SAP f) Hadoop g) Other What do you believe is the hardest part when it comes to use operational data within IT Systems? a) Data Preparation b) Lack of context/metadata c) Business Case not well defined d) Performance e) Ease of access of Operational Data
What you will hear during this talk: Why IT-OT convergence? PI Integrator for Business Analytics: Product details SAP HANA IoT Integrator by OSIsoft Frank Ruland Streaming pattern Roadmap Q&A 4
User Interfaces Data / Asset The Convergence of Information and Operational Technology SCADA HMI MES ERP Esri PLM User Interfaces Operational Insights SCADA PLC PI Data Arch. LIMS OT Strategy Architecture Security Governance Hardware IT ERP Business Insights Big Data Unstr. Data Data / Asset Typical information landscape. OT Operation Technology Empowering Business in Real-Time. IT Information Technology
Complexity Problem Complexity Drives the Need for Integration Disparate assets or interacting one-by-one Interacting with common assets as a fleet System Optimization Monitoring Real-time visibility Process Optimization Real-time & historical view across any plant asset Benchmarking Fleet-wide performance comparison Large scale multi-variate analysis Business Intelligence Big Data Analytics Machine Learning 6
How can I do this? Predict Outages Production Forecasting Estimate RUL Root-Cause Analysis Shift Analysis Compare asset performance Fleetwide BI reports Material Management Predictive Maintenance 7
Is it a smooth Journey? o Knowledge o Time o Support o Technology first o Scope o Flexibility More than 50 % of Big Data projects are unsuccessful! 8
Getting process data analysis-ready Collect Collecting high-fidelity high-frequency data from a variety of sources and systems Enhance Wrapping a layer of context around the data, assets and events OT Calculate Enriching the raw data by calculating KPIs, aggregations and different analysis Data Preparation feels like IT Correlate Finding patterns and relationships between variables across multiple datasets and datatypes Apply Algorithms Using statistics and machine learning to find insights in data across multiple variables
Data Scientist is the sexiest job of 21 st century, but Source: http://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/#5481f6037f75 10
Turbine 2 Turbine 1 Time series Data is.complex! Time Different Archive Start Times Comm Failure Speed Torque Bearing Temp Oil Temp Manufacturer Last Service Vestas June 20, 2013 Spike / Out of Range Speed Torque Additional Measure Bearing Temp Oil Temp Bad Sensor Wear Factor Manufacturer Siemens Last Service Uneven Spacing 11
What do we need to approach this problem? Collect Collecting high-fidelity high-frequency data from a variety of sources and systems OT IT Enhance Calculate Shape Align & Cleanse Augment Transport Correlate Apply Algorithms Wrapping a layer of context around the data Enriching the raw data by calculating KPIs, aggregations and different analysis Building a data model to respond a specific business question or analysis Synchronize multiple data sources so they are comparable and purify the raw data using filters Increase the information content by adding statistics and summary calculations Transform the time-series data into row-column format and push it right to the desired tools Finding patterns and relationships in data sets that aren t revealed in one data set Using statistics and machine learning to find insights in data across multiple variables
PI Integrator for Business Analytics/SAP/Azure Quality / Validate PULL Increase informatio n content CLEANSE AUGMENT Send the information directly to the tool SHAPE TRANSMIT Model your data structure PUSH 13
Demo: how are our compressors performing? Situation: We are tracking compressor process data and we are able to track downtimes as they happen Problem: We want to compare different compressors in terms of downtimes and understand which ones are offsetting from the baseline. We want to predict potential new outages to maximize asset availability Specific Capabilities: Prepare and craft data model for Business Intelligence on downtimes, answer a set of questions in terms of asset performance. Prepare and craft data model for Machine Learning analysis, bring the predictions back to PI and analyze the predictions to identify potential downtimes.
Demo: how are our compressors performing? 15
Demo Recap I was able to: Build a data model to answer a specific question Provide Large amount of information in context Quickly consume the data in BI/Machine Learning and get results Key Benefits: CAST Self-Service Performance Supports for multiple targets 16
Operational Reporting & Analysis Architecture PI Integrator for BA: Business Intelligence Edition Visualization & Analytics Data Preparation and Integration Layer System of Record Tablea u SAS Spotfir MSFT BI e All BI tools that support ODBC PI Integrator for Business Analytics Business Intelligence Edition PI Server I want to analyze operations data stored in the PI System using modern BI tools 17 17
Enterprise Data Warehouse Architecture PI Integrator for BA: Data Warehouse Edition Visualization & Analytics Enterprise Data Warehouse / Data Mart / Data Lake Data Preparation and Integration Layer Tableau PI Integrator for Business Analytics Spotfire SAS Oracle DW, SQL Server, Teradata MSFT BI Custom Applications Hadoop Custom or 3 rd Party Data Management and ETL I need to fit operational data into my existing company IT information architecture System of Record PI Server CRM Sales EAM ERP HR 18 18
Integration with SAP HANA: High level Architecture SAP HANA IoT Integrator by OSIsoft Visualization, Analytics, & Business Process Applications Lumira BOBJ Partner Applications SAP LOB Solutions & Applications GIS Enterprise Analytics and Applications Platform SAP HANA SAP BW Data Preparation and Integration Layer SAP HANA IoT Integrator by OSIsoft PI Integrator Framework SAP HANA Enterprise Information Management and Data Provisioning Agent Systems of Record Aggregation PI Server CRM Sales EAM ERP HR 19
SAP HANA IoT Integrator by OSIsoft Frank Ruland, SAP 20
Solution architecture - Pull PI data into SAP HANA via SAP HANA Smart Data Access SAP HANA HANA SDI SAP DP Agent Windows Aggregation PI Server 3 Windows (All Java) Web UI Shape Designer PI Data Archive Linux PI Asset Framework (AF) HANA Studio SAP HANA IoT Integrator by OSIsoft 1 PI JDBC PI Integrator Framework Microsoft SQL PI SQL DAS PI View 5 4 2 Windows 1 2 3 4 Note:- This is the heart of the Integrator. Prepare the time series data via CAST into a row/column format for consumption in a relational dbase environment User creates PI View in Web UI Shape Designer via PI Integrator Framework PI View definition is stored in PI System (AF). PI View data is stored in optimized format in AF-managed SQL Server SAP HANA user configures virtual tables in SAP HANA Studio using SAP HANA SDI and SAP HANA IoT Integrator by OSIsoft PI SQL DAS controls access to PI Views Legend: Component sold by SAP Component sold by OSIsoft Included with SAP HANA IoT Integrator by OSIsoft (no fee). Provisioned by OSIsoft 5 SAP HANA IoT Integrator by OSIsoft retrieves data from PI View located in SQL Server via PI JDBC driver
Solution architecture - Push PI data into SAP HANA via SAP HANA Client SAP HANA Note:- This is the heart of the Integrator. Prepare the time series data via CAST into a row/column format for consumption in a relational dbase environment HANA Client Windows (ODBC) 2 Linux 1 User creates PI View in Web UI Shape Designer via PI Integrator Framework Windows Web UI Shape Designer 1 PI Integrator Framework Aggregation PI Server PI View 2 PI Integrator Framework pushes data to HANA via HANA Client (ODBC) PI Data Archive PI Asset Framework Microsoft SQL Windows Legend: Component sold by SAP Component sold by OSIsoft Included with SAP HANA IoT Integrator by OSIsoft (no fee). Provisioned by OSIsoft
SAP HANA Real-time in-memory predictive analytics platform** Streaming Algorithms* Adaptive Hoeffding Tree Denstream Data-at-rest Algorithms Association Analysis Cluster Analysis Classification Analysis Time Series Analysis +60 Native Algorithms Streaming Engine In-memory Processing Engines Graph Engine Spatial Engine Calculation Engine Text Engine PAL APL AFL R Scripts SAP HANA In-memory In-database Predictive Analytics R Engine SAP HANA Studio Application Function Modeler Predictive Analysis Library (PAL) Automated Predictive Library (APL) Application Function Library (AFL) R integration Accelerated predictive analysis and scoring with native in-database algorithms for both data-atrest and for streaming data The predictive analysis capabilities of SAP Predictive Analytics automated analytics engine (formerly KXEN / II) in SAP HANA Application Function Library (AFL) framework allows SAP, partner, and customers to develop, deploy, load, and leverage their own advanced analytic custom functions in SAP HANA Execution of R scripts via high- performing parallelized vector based connection; R scripts embedded as part of overall query plan SAP Custom Open Source * Predictive Algorithms for Streaming come with Smart Data Streaming License
SAP HANA In-Memory Predictive Analytics Combine the depth and power of in-memory analytics within SAP HANA with the breadth of R to support a variety of advanced analytic and predictive scenarios Predictive Analysis Library (PAL) Native predictive algorithms In-database processing for powerful and fast results Quicker implementations Support for clustering, classification, association, time series etc R Integration for SAP HANA Enables the use of the R open source environment (> 3,500 packages) in the context of the HANA in-memory database R integration enabled via high performing parallelized connection R script is embedded within SAP HANA SQL Script
Compressor Demo video
Product Roadmap Matt Ziegler, OSIsoft 26
Complexity Problem Complexity Drives the Need for Integrators Disparate assets or interacting one-by-one Interacting with common assets as a fleet System Optimization Monitoring Real-time visibility Process Optimization Real-time & historical view across any plant asset Benchmarking Fleet-wide performance comparison Large scale multi-variate analysis HMI PI ProcessBook PI Coresight PI Datalink BI App (i.e. Tableau, Spotfire, Lumira) PI Integrator for Business Analytics PI Integrator for SAP HANA Machine Learning (Azure ML, R) PI Integrator for Business Analytics PI Integrator for SAP HANA 27
More integration options, more systems 2015 1H-2016 Future Business Intelligence & Data Warehouses Available Today Scalable BI for the PI System v1.0 Fleet Asset Reporting Reduce Reporting Time Integrate w/ Data Warehouse Available Today Expanded Systems and Events v1.1 + Oracle + Hadoop (HIVE & HDFS) Event Frames Planned (2H 2016) Cloud Platforms Microsoft Azure Azure SQL, SQL DW Azure Data Lake SAP HANA Cloud Platform Streaming Systems Research Streaming Pattern Enabling computations in realtime with an external compute engine Planned (1H 2017) Stream Systems Azure Event & IoT Hub Kafka Custom json Research Partner Platform Enable partners and customers to build applications and interact programmatically using PI Integrator Framework
Tables vs Streams Business Intelligence Human readable Batch / Bulk Process Normalized data Regularly scheduled Large data, few messages In-line (Streaming) Analytics Computer readable Specific Data / Targeted Process Raw or Packages of data Triggered Small data, many messages 29
Integration Patterns Tables Streams Other Patterns Metadata Programming On-Demand Workflow & Transactions Files Databases Files Queues Messaging External Analytics Engines 30
2016 Technical Roadmap - Specific Planned Enhancements SAP HANA IoT Integrator by OSIsoft Version 1 (Dec 15) Version 1.5 (May 16) Version 2 (Q4 16) Batch-cleanse, filter and aggregate PI System data into federated tables within SAP HANA using SDA Leverage event frames (batches) of PI System data published into SAP HANA for ad hoc projects and analysis in memory Publish PI System data to SAP HANA or HANA Cloud Platform using SDI. Retrieve live information from PI Views using SDI. Protect your technology investment!
Example Roadmap - SAP HANA IoT Integrator by OSIsoft Access Road Map Databases & Applications Query Data (API) Pull Data SAP IoT Integrator 2016 SAP HANA IoT Integrator 2015 (Smart Data Access) Virtual Tables in HANA Lumira S/4 HANA PdMS Analytics Publish Data (Push) Stream Data (Stream Events-SDS) SAP HANA IoT Integrator 2015 SP1 2016 (SDI) Planned 2017 Smart Data Streaming SAP HANA Ecosystem Receive Data (Predictions) Planned 2017 SAP HANA / HANA Cloud Platform Receive Metadata (Assets / PM) Research S4 (PM) AIN 32
Contact Information Franco Camba fcamba@osisoft.com Systems Engineer OSIsoft ltd UK Matt Ziegler mziegler@osisoft.com Product Manager OSIsoft, LLC Frank Ruland frank.ruland@sap.com Head of Industry Ecosystem for Energy and Natural Resources SAP SE 33
Questions Please wait for the microphone before asking your questions Please remember to Complete the Online Survey for this session State your name & company http://ddut.ch/osisoft 34
Thank You