Introducing Microsoft SQL Server 2016 R Services. Julian Lee Advanced Analytics Lead Global Black Belt Asia Timezone

Transcription

1 Introducing Microsoft SQL Server 2016 R Services Julian Lee Advanced Analytics Lead Global Black Belt Asia Timezone

2 SQL Server 2016: Everything built-in built-in built-in built-in built-in built-in $2, SQL Server SQL Server SQL Server #1 #2 #3 Oracle is #5 $480 $120 Microsoft Tableau Oracle SQL Server Oracle MySQL SAP HANA TPC-H Self-service BI per user at massive scale In-memory across all workloads Consistent experience from on-premises to cloud 2

3 From data to decisions and actions Data $1.6 trillion Decisions Action Value

4 Microsoft Advanced Analytics Offerings Cortana Analytics Suite SQL Server 2016

5 Typical advanced analytics lifecycle Preparation Modeling Model Ingest Transform Explore Model Deploy ƒ(x) Operationalization Score Score Visualize Measure

6 Data Scientist should be creating / testing models Preparation Modeling Model Ingest Transform Explore Model Deploy ƒ(x) Operationalization Score Score Visualize Measure

7 But the reality is different Data scientist focus time Preparation 80% 15% Modeling Model Ingest Transform Explore Model Deploy ƒ(x) 5% Operationalization Score Score Visualize Measure

8 R What is it? Open source lingua franca Global community Ecosystem Big data Analytics, computing, modeling Millions of users 7,000+Packages Scalability

9 CRAN: The Comprehensive R Archive Network Open Source lingua franca Analytics, Computing, Modeling In addition to CRAN, Bioconductor, GitHub, and others distribute R packages

10 R Why use it? Large talent base knows how to use it already Dividing use between roles creates efficiency Protecting important data is much easier Scalability for ongoing computation of data

11 Challenges posed by open source R $? Uncertain total cost of ownership and return on investment Integrating R with existing and ever changing data infrastructures Scale and Performance Data movement restricts access for efficient data modeling

12 SQL Server 2016 R Services offers 24/7 support, enterprise-grade platform Included in SQL 2016 Single Analytics platform on multiple data infrastructures Smarter decisions, faster analysis and results Hybrid cloud, reusable code, fewer limits

13 In-database advanced analytics Extensibility Example solutions Real-time operational analytics without moving data Data Scientist Interacts directly with data R integration? Sales forecasting Warehouse efficiency Predictive maintenance Credit risk protection Analytics library R with in-memory scalability SQL Developer/DBA Manage data and analytics together T-SQL interface Relational data Built into SQL Server 2016

14 SQL Server R Services: Enterprise R Analytics SQL 2016 Prepare Model Operationalize Typical Advanced Analytics Process Model, Deploy & Operationize In SQL16: Support Entire Analytics Lifecycle Enable R Users to Run R Inside SQL 2016 Enable SQL Users to Extend BI Applications Using R Analytics Advantages: Scale By Eliminating Movement Scale Using Parallelized Analytics Reduced Security Exposure SQL Skill Reuse for Data Engineering SQL Skill Reuse for App development Improved Operational Stability for Applications

15 Improving advanced analytics with R and SQL Server Data Scientist Publish algorithms and interact directly with data Run R across entire data set Execute and test in-database Deploy to local database SQL Developer Better manage data and analytics together Operationalize R script/model Use T-SQL constructs DBA Better manage storage and analytics together More easily maintain performance and stability Secure and govern R runtime execution

16 Parallelized, remote executing algorithms Data step Data import Delimited, fixed, SAS, SPSS, OBDC Variable creation & transformation Recode variables Factor variables Missing value handling Sort, merge, split Aggregate by category (means, sums) Descriptive statistics Min/max, mean, median (approx.) Quantiles (approx.) Standard deviation Variance Correlation Covariance Sum of squares (cross-product matrix for set variables) Pairwise cross tabs Risk ratio & odds ratio Cross-tabulation of data (standard tables & long form) Marginal summaries of cross tabulations Statistical tests Chi Square Test Kendall Rank Correlation Fisher s Exact Test Student s t-test Sampling Subsample (observations & variables) Random sampling Predictive models Sum of squares (cross-product matrix for set variables) Multiple linear regression Generalized linear models (GLM) exponential family distributions: binomial, Gaussian, inverse Gaussian, Poisson, Tweedie. Standard link functions: cauchit, identity, log, logit, probit. User defined distributions & link functions. Covariance & correlation matrices Logistic regression Classification & regression trees Predictions/scoring for models Residuals for all models Custom parallelization PEMA-R API rxdatastep rxexec Simulation Simulation (e.g., Monte Carlo) Parallel random number generation K-Means Cluster analysis Classification Decision trees Decision forests Gradient-boosted decision trees Naïve Bayes

17 Faster and more scalable

18 Appendix

19 Want to know more? Find out how much R knowledge is already in-house Contact your Microsoft Data Platform Specialist or local Black Belt Learn more about the relationship between Microsoft and Revolution Analytics

20 5 Key Hurdles for Advanced Analytics Load Data Build Insight Re-Code? = ( ) $ $ $ Rp? Innovation Rate Data Engineer Data Scientist Analyst Business Executive Large Unknown Current Spend Decisions Data Lake Developer Data Steward Applications Rapid Big Data Evolution MapReduce YARN Spark? < 3 years Expanding Communities Complex Infrastructure Decisions Fit Risk Agility? Security Elasticity Experience TCO Transition Cost Cloud On-Prem

21 Demo Link