Azure SQL Data Warehouse Andrija Marcic Microsoft
End to end platform built for the cloud Hadoop SQL Azure SQL Data Warehouse Azure SQL Database App Service Intelligent App Azure Machine Learning Power BI
SQL Server, Azure VMs, Azure SQL DB On Premises Hybrid Cloud Options: Shared Lower Cost Virtual Infrastructure as a service SQL Platform as a service SQL SQL Server in Azure VM Virtualized machines Software as a services Azure SQL Database Virtualized Databases SQL Server on physical machines SQL Server in on-premises VM (private cloud) SQL Server in Azure VM (public cloud) Azure SQL Database (public cloud) Physical SQL Server Private Cloud Virtualized Machines + Appliances Dedicated Higher Cost SQL SQL Server Physical Machines(raw Iron) Off Premises Higher Administration Lower Administration
Demo
Azure SQL DB vs Azure SQL DataWarehouse
Azure SQL DB
SQL Azure Data Warehouse (MPP)
SQL Server
Real-time elasticity Expand or reduce as needed Resize in <1 minute On-demand compute
Data Warehouse Units (DWUs) are a measure of reserved compute performance or power. A customer s DWU needs can vary depending on the needs of their workload. 100 200 300 400 500 600 1000 1200 1500 2000 3000 6000 DATA WAREHOUSE UNITS (DWUS)
DWU DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 DW3000 DW6000 ALTER DATABASE ContosoDW MODIFY (service_objective = 'DW1000' ) ;
Data Warehouse Unit (DWU)
Demo
CREATE DATABASE MyDB COLLATE SQL_Latin1_General_CP1_CI_AS ( EDITION = 'DataWarehouse', SERVICE_OBJECTIVE = 'DW400', MAXSIZE = 10240 GB ); ALTER DATABASE MyDB MODIFY (MAXSIZE = 245760 GB);
SQL Server Bulk Copy tool, SQL Server Integration Services (SSIS), Informatica, Attunity, Bryte Systems, snaplogic, others Azure Blob Storage, HDInsight (future), Azure Data Lake Store (future) Azure Data Factory, Redgate Data Platform Studio
Query unstructured data via PolyBase/T-SQL SQL DW instance Scale out compute PolyBase Hadoop VMs / Azure Storage/ Azure Datalake Any data, any size, anywhere
Quote: ************************ ********************** T-SQL query ********************* ********************** *********************** $658.39 Azure SQL DW Hadoop Name DOB State Jim Gray 11/13/58 WA Ann Smith 04/29/76 ME
open sessions 1024 32 active queries
Slots 128 64 32 16 8 4 2 1 mediumrc (m) largerc (l) xlargerc (xl) Max Concurrency Slots
% Consumed 100,0 90,0 80,0 70,0 60,0 50,0 40,0 30,0 20,0 10,0 0,0 mediumrc (m) largerc (l) xlargerc (xl) DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000
Slots 32 28 24 20 16 12 8 4 0 DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 Concurrency impact
DWU Max External Readers Max Writers DW100 8 60 DW200 16 60 DW300 24 60 DW400 32 60 DW500 40 60 DW600 48 60 DW1000 80 80 DW1200 96 96 DW1500 120 120 DW2000 160 160 DW3000 240 240 DW6000 480 480
8000 7000 GB/Hour 6000 5000 4000 3000 2000 1000 0 1 2 3 4 5 6 10 12 15 20 30 60 GB/Hour
25000 20000 15000 10000 5000 0 DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 DW3000 DW6000 Capacity (GB)
Recommended starting point Flexibility to select any range of DWUs >160 TBs 80-160 TBs 60-80 TBs 48-60 TBs 36-48 TBs 20-36 TBs 16-20 TBs 12-16 TBs 8-12 TBs 4-8 TBs 0-4 TBs 100 200 300 400 500 600 1000 1200 1500 2000 3000 6000
?
Tack för eran uppmärkasmhet
Tack för eran uppmärksamhet & Glöm inte att fylla i utvärderingen
Azure SQL Data Warehouse Architecture Application or User connection Data Loading (SSIS, REST, OLE, ADO, ODBC, WebHDFS, AZCopy, PS) SQL DB Compute SQL DB DMS Compute Control Massively Parallel Processing (MPP) Engine DMS DMS DMS DMS SQL DB Compute DMS (Data Movement Service) executes across all database nodes SQL DB Compute Storage and Compute are de-coupled, enabling a true elastic service and separate charging for both compute and storage 100 DWU < > 2000 DWU Compute Scale compute up or down when required (SLA <= 60 seconds). Pause, Restart, Stop, Start. HDInsight Blob storage [WASB(S)] Azure Infrastructure and Storage Storage Add\Load data to WASB(S) without incurring compute costs
Azure SQL Data Warehouse Control Control SQL DB Massively Parallel Processing (MPP) Engine Control SQL DB SQL DB SQL DB SQL DB Compute Compute Compute Compute HDInsight Blob storage [WASB(S)] Endpoint for connections Regular SQL endpoint (TCP 1433) Persists no user data (metadata only) Coordinates compute activity using MPP
Azure SQL Data Warehouse - Compute s Control Massively Parallel Processing (MPP) Engine SQL DB Compute (s) SQL DB SQL DB SQL DB SQL DB Compute Compute Compute Compute Azure SQL Database Blob storage [WASB(S)] An increase of DWU will increase the number of compute nodes HDInsight
Azure SQL Data Warehouse Blob storage Control Massively Parallel Processing (MPP) Engine SQL DB Compute SQL DB Compute SQL DB Compute SQL DB Compute RA-GRS storage +PB s of storage Load data without incurring compute costs Blob storage [WASB(S)] HDInsight