Modeling. Preparation. Operationalization. Profile Explore. Model Testing & Validation. Feature & Algorithm Selection. Transform Cleanse Denormalize

Transcription

1

2

3 Preparation Modeling Ingest Transform Cleanse Denormalize Profile Explore Visualize Feature & Algorithm Selection Model Testing & Validation Operationalization Models Visualizations Deploy Apps, Services & Visualizaitons Act: Score, Visualize Measure

4

5 Advanced Analytics Portfolio Comprehensive Connected Choice R Server SSRS SQL Server Integration Services SQL Server Analysis Services Datazen SQL Server MDS, DQS SharePoint BI HIVE PIG OOZIE SQOOP SQL SQL Server Hadoop (on-premise / Azure VM) HBase Excel BI Storm Applications On-premise Hadoop/OS 5

6 Intelligence Cognitive Services, Bot Framework. Cortana Dashboards and Visualizations Power BI SSRS, DataZen, R R visualizations Machine learning and Analytics Machine learning, Stream Analytics, HDInsight SSAS, SQL Server R Services Microsoft R Big Data Store Data Lake, SQL, DW OLTP, DW, Hadoop, EDSs Hadoop, Teradata, Linux, Windows Information Management Data Factory, Data Catalog, Event Hubs SSIS, DQS, MDS Spark SQL/ETL Cortana Intelligence Suite SQL Server 2016 Microsoft R Cloud analytics In Database analytics R-based analytics

7 Transform data into intelligent action Preconfigured Solutions Dashboards and Visualizations Machine Learning and Analytics Big Data Store Information Management Intelligence Cognitive Services Bot Framework Cortana

8 Easily build, deploy, and share predictive analytics solutions Machine Learning and Analytics Machine Learning Data Lake Analytics HDInsight (Hadoop and Spark) Stream Analytics Simple, scalable, cutting edge. A fully managed cloud service that enables you to easily build, deploy, and share predictive analytics solutions. Deploy in minutes. Azure Machine Learning means business. You can deploy your model into production as a web service that can be called from any device, anywhere and that can use any data source. Publish, share, monetize. Share your solution with the world in the Gallery or on the Azure Marketplace.

9 Demo

10

11 Challenges posed by open source R?? Limited Data Scale in Memory Singlethreaded / Not parallelized Lack of Commercial Support Complex Deployment Processes Production

12 Microsoft R R products Products Microsoft R Open / Client Free and open source R distribution Enhanced and distributed by Microsoft SQL Server R Services Built in Advanced Analytics and Stand Alone Server Capability Leverages the Benefits of SQL 2016 Enterprise Edition Microsoft R Server Microsoft R Server for Redhat Linux Microsoft R Server for SUSE Linux Microsoft R Server for Teradata DB Microsoft R Server for Hadoop on Redhat

13 Microsoft R Server MRS extends open-source R to allow: Multi-threading Matrix operations, linear algebra, and many other math operations run on all available cores Parallel processing ScaleR functions utilize all available resources, local or distributed On-disk data storage RAM limitations lifted

14 MRS Components Platforms R+CRAN Open source R interpreter R Freely-available huge range of R algorithms Algorithms callable by RevoR Embeddable in R scripts 100% Compatible with existing R scripts, functions and packages RevoR Performance enhanced R interpreter Based on open source R Adds high-performance math library to speed up linear algebra functions R Open Microsoft R Server DeployR ScaleR Ready-to-Use high-performance big data big analytics Fully-parallelized analytics Data prep & data distillation Descriptive statistics & statistical tests Range of predictive functions User tools for distributing customized R algorithms across nodes Wide data sets supported thousands of variables ConnectR High-speed & direct connectors Available for: High-performance XDF SAS, SPSS, delimited & fixed format text data files Hadoop HDFS (text & XDF) Teradata Database & Aster EDWs and ADWs ODBC DistributedR Distributed computing framework Delivers cross-platform portability

15 Demo

16 Flexibility & Agility Manage and Monitoring Performance & Scale Cost Effective

17 Model Development (R Users) Working from R IDE on a local workstation, execute an R script that runs in-database on remote SQL server, and get the results back. sqlcompute <- RxInSqlServer() rxsetcomputecontext(sqlcompute) linmodobj <- rxlinmod() Data Scientist Workstation 1 Script SQL Server Execution R IDE 3 Results Advanced Analytics Extensions Microsoft R Open Microsoft R Open Microsoft R Server Microsoft R Server

18 Model Operationalization with SQL R Services R Code->T-SQL Stored Proc Call a T-SQL System Stored Procedure to generate features and train (or retrain) the model Call a T-SQL System Stored Procedure from my application and have it trigger R script execution indatabase to predict on new dataset. Results are then returned to my application (predictions, plots). Application 1 3 Call System Stored Procedure Results: scores, plots SQL Server 2016 Advanced Analytics Extensions Microsoft R Open 2 The stored procedure contains R code and executes in-database exec = = -- R code -- Mcirosoft R Server

19 R script usage from SQL Server Original R script: IrisPredict <- function(data, model){ library(e1071) predicted_species <- predict(model, data) return(predicted_species) } library(rodbc) conn <- odbcconnect("mysqlazure", uid = myuser, pwd = mypassword); Iris_data <-sqlfetch(conn, "Iris_Data"); Iris_model <-sqlquery(conn, "select model from my_iris_model"); IrisPredict (Iris_data, model); Calling R script from SQL Server: /* Input table schema */ create table Iris_Data (name varchar(100), length int, width int); /* Model table schema */ create table my_iris_model (model varbinary(max)); varbinary(max) = (select model from my_iris_model); exec = = ' IrisPredict <- function(data, model){ library(e1071) predicted_species <- predict(model, data) return(predicted_species) } IrisPredict(input_data_1, model); = = N'select * from = N'@model with result sets ((name varchar(100), length int, width int, species varchar(30))); Values highlighted in yellow are SQL queries embedded in the original R script Values highlighted in aqua are R variables that bind to SQL variables by name Advanced analytics

20 R integration and advanced analytics SQL Server Analytics library Share and collaborate Manage and deploy Analytical engines Integrate with R Become fully extensible R + Data Scientists Publish algorithms, interact directly with data DBAs Manage storage and analytics together Capability Extensible in-database analytics, integrated with R, exposed through T-SQL Centralized enterprise library for analytic models Benefits Data management layer Relational data Use T-SQL interface Stream data in-memory Business Analysts Analyze through T-SQL, tools, and vetted algorithms Advanced analytics

21 Summary: Advanced Analytics Products Intelligence Cognitive Services, Bot Framework. Cortana Dashboards and Visualizations Power BI SSRS, DataZen, R R visualizations Machine learning and Analytics Machine learning, Stream Analytics, HDInsight SSAS, SQL Server R Services Microsoft R Big Data Store Data Lake, SQL, DW OLTP, DW, Hadoop, EDSs Hadoop, Teradata, Linux, Windows Information Management Data Factory, Data Catalog, Event Hubs SSIS, DQS, MDS Spark SQL/ETL Cortana Intelligence Suite SQL Server 2016 Microsoft R Cloud analytics In Database analytics R-based analytics

22 Microsoft Cognitive Services Give your apps a human side

23 Cognitive Services API Collection Microsoft Cognitive Services Give your apps a human side

24

25 Rolls-Royce Filtering the signal from the noise Objectives Connect Rolls-Royce jet engine data to Microsoft's intelligent cloud for insights to improve aircraft performance, safety and maintenance. Tactics Use Cortana Intelligence to scale quickly and efficiently, aggregate data across customer fleets and process data in real time. Results More efficient flight and maintenance plans Targeted and actionable fuel efficiency insights Quickly-generated reports and dashboards that tell compelling stories and deliver high-quality insights Our goal is not data for the sake of data, but to embrace the cloud and analytical technologies to deliver more expert insights to the right stakeholders at the right time. Nick Farrant Senior Vice President ROLLS-ROYCE

26 Microsoft and Rolls-Royce collaborate to offer advanced operational intelligence to airlines Solution Rolls Royce uses Cortana Intelligence to connect jet engine data to Microsoft's Cloud to improve aircraft performance, safety and maintenance Results Drive 1-2% increase in fuel efficiency, saving $Ms/year. Use savings from actionable insight to offer additional services to customers. Do failure prediction for next flights and fleet monitoring using anomaly detection. DEMO Using our energy management platform based on Azure, they can cut those peak hours, avoid the upgrade, and save a lot of money. - Knut Johansen, CEO