Streaming iphone sensor data to SAS Event Stream Processing

Transcription

1 SAS USER FORUM Streaming iphone sensor data to SAS Event Stream Processing Pasi Helenius Senior Advisor

2 SAS Event Stream Processing 3 KEY CHARACTERISTICS Technology Process steams of data events, on the move, prior to storage, when events happen Speed Actionable Intelligence Process huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisecond) Filters/aggregates/correlates stream(s) to focus and detect specific events, patterns or characteristics, that help the business

3 New era of information processing USERS NEED IMMEDIATE Answers Streaming Analytics Streaming Move analysis to event source Analyze before data is stored Keep what is relevant Micro-Batch Batch Micro-seconds Time to decision Processing streaming data is about getting immediate answers to reduce time to decision Days

4 Smart Cities and Homes Connected Customer Communications Surveillance Connected Car/ Transportation Energy I OFnternet T hings Building Management Agriculture Insurance Manufacturing Healthcare Retail

5 I O T The IOT Promise B I G DATA High Velocity Complex Large Analytics Act Efficiencies New Value Quality of Life Early Warnings New Business Models Sense Understand Act

6 Traditional Analytics Lifecycle Data Data Storage f ETL Deploy Alerts / Reports Access Store - Analyze

7 Enrich Store Deplo y Streaming Analytics Lifecycle Stream Understand Act Data Data Storage ETL Train Score Deploy Alerts - Reports Decisioning Streaming Data Streaming Model Execution Supervise Train Score

8 Publish Subscribe SAS Event Stream Processing Streaming Events ENGINEERED FOR FAST AND ADAPTIVE ACTION SAS Event Stream Processing Model Event Actions Continuous Query SAS In-Memory Pattern detection at event stream source Offline, data at rest identifies emerging trends Feedback new insights back into event streams Dynamically update queries in-stream Viya-enabled SAS-generated Insights Enrichment Data Analytic Models Business Rules Copyr i g ht 2015, SAS Ins titut e Inc. All rights res er ve d.

9 SAS Event Stream Processing Engineered for AGILITY Low footprint OS native application From lightweight embedded technology to cloud distributed architecture Fulfill new IoT architecture needs Lightweight embedding technology Cloud ready OS native application Clustering Dynamic model update Edge Small Large Cluster Cloud

10 Edge-to-Enterprise IoT Analytics Platform Cisco and SAS

11 SAS Event Stream Processing IOT Scenario SAS ESP Server ESP for Edge Computing SAS ESP Analytics SAS ESP Server SAS ESP Studio SAS Event Stream Manager SAS Streamviewer SAS ESP Analytics

12 Demo

13 Demo

14 ESP Model showing in ESP model Setting retention to get previous N readings Join the moving average with current reading Calculate the average acceleration of X,Y,Z axials

15 SAS Event Stream Processing Advanced Analytics

16 Enrich Store Deploy Streaming Analytics Lifecycle Stream Understand Act Data Data Storage ETL Train Score Deploy Alerts - Reports Decisioning Streaming Data Streaming Model Execution Supervise Train Score SWEDEN 2017

17 SAS Event Stream Processing Learning Models and Feedback Loop Machine Learning streaming algorithm support DBSCAN (density-based clustering) K-Means ESP Studio Model authoring assistant More algorithms and a monitoring windows to be added in future releases

18 SAS Event Stream Processing High End Streaming Analytics Temporal pattern detection and analysis Business rules data quality and policy definitions Filter, aggregate and correlate events Text analytics Streaming geofencing Reference historic data Lambda architecture In-Stream analytic models processing SAS Model Manager 9.4 Integration SAS Datastep, SAS DS2, Python, C SAS ASTORE Scoring support *SAS Event Stream Processing 5.1 Streaming Algorithms Streaming Summary - Univariate Statistics Streaming Pearson s Correlation Streaming Segmented Correlation Weibull Distribution Fitting Short Time Fourier Transform Streaming Text Tokenization Out-of-Stream Training Random Forest Gradient Boosting Tree Factorization Machine Support Vector Machine Support Vector Data Description In-Stream Training Streaming K-Means Streaming DBSCAN Streaming Linear Regression* Streaming Support Vector Machines* Streaming Logistic Regression*

19 New ESP Window Types Machine learning and high frequency analytics support Calculate Window Streaming Univariate Statistics and Correlation Train Window/Score Window Streaming K-Means and DBSCAN, ASTORE Support, Summary, Text, Fourier Transforms confi g data data Calculat e score data confi g Coordination with Model Reader and Train window to control runtime model deployment Queue-based model management Model Supervisor Window Model Reader Window Inject models to Score window, e.g., ASTORE signal Train model Model Supervisor model model model confi g Score Model Reader score model

20 SAS ESP: Text Analytics Scoring In-stream unstructured text analysis Detect interesting events in unstructured text data streams Define taxonomies/models within SAS Text Analytics suite Process streams with dedicated ESP windows Extract concepts and categorize content Text Context & Text Category windows Analyze sentiment Text Sentiment window Future: SAS ESP 4.3: Release 17w21 New Text Topics window Text Topic window runs Text Mine actions on events

21 SAS Event Stream Processing 3 KEY CHARACTERISTICS Technology Process steams of data events, on the move, prior to storage, when events happen Speed Actionable Intelligence Process huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisecond) Filters/aggregates/correlates stream(s) to focus and detect specific events, patterns or characteristics, that help the business

22 SAS USER FORUM Thank You!