Azure SQL Data Warehouse. Andrija Marcic Microsoft

Size: px

Start display at page:

Download "Azure SQL Data Warehouse. Andrija Marcic Microsoft"

Kory Johns
6 years ago
Views:

1 Azure SQL Data Warehouse Andrija Marcic Microsoft

2 End to end platform built for the cloud Hadoop SQL Azure SQL Data Warehouse Azure SQL Database App Service Intelligent App Azure Machine Learning Power BI

3 SQL Server, Azure VMs, Azure SQL DB On Premises Hybrid Cloud Options: Shared Lower Cost Virtual Infrastructure as a service SQL Platform as a service SQL SQL Server in Azure VM Virtualized machines Software as a services Azure SQL Database Virtualized Databases SQL Server on physical machines SQL Server in on-premises VM (private cloud) SQL Server in Azure VM (public cloud) Azure SQL Database (public cloud) Physical SQL Server Private Cloud Virtualized Machines + Appliances Dedicated Higher Cost SQL SQL Server Physical Machines(raw Iron) Off Premises Higher Administration Lower Administration

4 Demo

5 Azure SQL DB vs Azure SQL DataWarehouse

6 Azure SQL DB

7 SQL Azure Data Warehouse (MPP)

8 SQL Server

10 Real-time elasticity Expand or reduce as needed Resize in <1 minute On-demand compute

11 Data Warehouse Units (DWUs) are a measure of reserved compute performance or power. A customer s DWU needs can vary depending on the needs of their workload DATA WAREHOUSE UNITS (DWUS)

12 DWU DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 DW3000 DW6000 ALTER DATABASE ContosoDW MODIFY (service_objective = 'DW1000' ) ;

13 Data Warehouse Unit (DWU)

14 Demo

18 CREATE DATABASE MyDB COLLATE SQL_Latin1_General_CP1_CI_AS ( EDITION = 'DataWarehouse', SERVICE_OBJECTIVE = 'DW400', MAXSIZE = GB ); ALTER DATABASE MyDB MODIFY (MAXSIZE = GB);

21 SQL Server Bulk Copy tool, SQL Server Integration Services (SSIS), Informatica, Attunity, Bryte Systems, snaplogic, others Azure Blob Storage, HDInsight (future), Azure Data Lake Store (future) Azure Data Factory, Redgate Data Platform Studio

22 Query unstructured data via PolyBase/T-SQL SQL DW instance Scale out compute PolyBase Hadoop VMs / Azure Storage/ Azure Datalake Any data, any size, anywhere

23 Quote: ************************ ********************** T-SQL query ********************* ********************** *********************** $ Azure SQL DW Hadoop Name DOB State Jim Gray 11/13/58 WA Ann Smith 04/29/76 ME

25 open sessions active queries

26 Slots mediumrc (m) largerc (l) xlargerc (xl) Max Concurrency Slots

27 % Consumed 100,0 90,0 80,0 70,0 60,0 50,0 40,0 30,0 20,0 10,0 0,0 mediumrc (m) largerc (l) xlargerc (xl) DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000

28 Slots DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 Concurrency impact

29 DWU Max External Readers Max Writers DW DW DW DW DW DW DW DW DW DW DW DW

30 GB/Hour GB/Hour

31 DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 DW3000 DW6000 Capacity (GB)

32 Recommended starting point Flexibility to select any range of DWUs >160 TBs TBs TBs TBs TBs TBs TBs TBs 8-12 TBs 4-8 TBs 0-4 TBs

33 ?

34 Tack för eran uppmärkasmhet

35 Tack för eran uppmärksamhet & Glöm inte att fylla i utvärderingen

36 Azure SQL Data Warehouse Architecture Application or User connection Data Loading (SSIS, REST, OLE, ADO, ODBC, WebHDFS, AZCopy, PS) SQL DB Compute SQL DB DMS Compute Control Massively Parallel Processing (MPP) Engine DMS DMS DMS DMS SQL DB Compute DMS (Data Movement Service) executes across all database nodes SQL DB Compute Storage and Compute are de-coupled, enabling a true elastic service and separate charging for both compute and storage 100 DWU < > 2000 DWU Compute Scale compute up or down when required (SLA <= 60 seconds). Pause, Restart, Stop, Start. HDInsight Blob storage [WASB(S)] Azure Infrastructure and Storage Storage Add\Load data to WASB(S) without incurring compute costs

37 Azure SQL Data Warehouse Control Control SQL DB Massively Parallel Processing (MPP) Engine Control SQL DB SQL DB SQL DB SQL DB Compute Compute Compute Compute HDInsight Blob storage [WASB(S)] Endpoint for connections Regular SQL endpoint (TCP 1433) Persists no user data (metadata only) Coordinates compute activity using MPP

38 Azure SQL Data Warehouse - Compute s Control Massively Parallel Processing (MPP) Engine SQL DB Compute (s) SQL DB SQL DB SQL DB SQL DB Compute Compute Compute Compute Azure SQL Database Blob storage [WASB(S)] An increase of DWU will increase the number of compute nodes HDInsight

39 Azure SQL Data Warehouse Blob storage Control Massively Parallel Processing (MPP) Engine SQL DB Compute SQL DB Compute SQL DB Compute SQL DB Compute RA-GRS storage +PB s of storage Load data without incurring compute costs Blob storage [WASB(S)] HDInsight

L300 deck. Paul Duffett Mar2017

L300 deck. Paul Duffett Mar2017 L300 deck Paul Duffett (paduffet@microsoft.com) Twitter: @paulduffett Mar2017 Contents In presentation mode click on any box to jump to that section Introduction Use case, pattern and adoption Scenarios