Azure-persistence MARTIN MUDRA

Size: px

Start display at page:

Download "Azure-persistence MARTIN MUDRA"

Francine Goodman
6 years ago
Views:

1 Azure-persistence MARTIN MUDRA

2 Storage service access Blobs Queues Tables

3 Storage service Horizontally scalable Zone Redundancy Accounts Based on Uri Pricing Calculator

4 Azure table storage Storage Account Account Table Cars Customers Entities Name= Color= Name= Color= Name= Surname=

5 Azure table storage II PartitionKey Load balancing across storage nodes Partition Range of entities with same key Up to 1KB RowKey Primary key PartitionKey + RowKey Up to 1KB TimeStamp DateTime Ignored on insert & update Optimistic concurrency Supported Types Binary byte[] Boolean bool DateTime DateTime Double double Guid Guid Int32 int Int64 long String string

6 Azure tables - Query Select Max 1000 items Max 5 sec Pagination Top Filter Max 15 discrete comparisons Output formats Atom Json v 3.0 Xml Supported filters Equal eq GreaterThan gt GreaterThanOrEqual ge LessThan lt LessThanOrEqual le NotEqual ne Supported Boolean operators And Not Or

7 Azure tables - transaction Batch operations Insert Update Delete OData protocol Same PartitionKey Max 100 operations

8 Azure tables - scalability Partition distribution across servers Health of the servers Traffic Load balancing Target 500 entities per second / partition DFS layer IO load Server busy / timeout error Retry strategy No retry Fixed backoff Exponential backoff

9 Azure tables scalability II PartitionKey Granularity Partition Size Advantages Disadvantages Single value Small number of entities Batch transactions are possible with any entity All entities are local and served from the same storage node Single value Large number of entities Entity Group Transactions may be possible with any entity. Multiple values There are multiple partitions Partition sizes depend on entity distribution Batch transactions are possible on some entities Dynamic partitioning is possible Single-request queries possible (no continuation tokens) Load balancing across more partition servers possible Unique values There are many small partitions. The table is highly scalable Range partitions may improve the performance of cross-partition range queries Scaling is limited. Throughput is limited to the performance of a single server. A highly uneven distribution of entities across partitions may limit the performance of the larger and more active partitions Queries that involve ranges may require visits to more than one server. Batch transactions are not possible. Append or prepend-only patterns can affect insert-throughput

10 Azure tables Scalability performance Query Type PartitionKey Match RowKey Match Performance Rating Point Exact Exact Best Row range scan Exact Partial Better with smaller-sized partitions Bad with very large-sized partitions Partition range scan Partial Partial Good with a small number of partition servers being touched Worse with more servers being touched Full table scan Partial, none Partial, none Worse with a subset of partitions being scanned Worst with all partitions being scanned

11 Azure blob Storage Account Container Blob Account Pictures IMG001.jpg IMG002.jpg Videos Video001.mpeg

12 Azure blob - Blob types Block blob Blocks Upload large Parallel upload Commit all at once Page blob Azure Premium storage Pages Commit by page Random reads/writes Ideal for VHD

13 Azure blob - Block blob Block Up to 4MB blocks BlockId Base64 string Insert Delete Modify One-week to commit blocks Up to uncommitted blocks (max 400GB) Transaction 64 MB in max size in single request Commit by block BlockBlob N

14 Azure blob Page blob Page 512 bytes Max size 1 TB Write Change Range & offset align to 512 bytes Writes in-place Pageblob N Commit immediately (by page)

15 SQL Server Existing application Disaster recovery Full admin rights On-premise resources Active Directory Eliminates hardware Hybrid cloud Pricing Calculator

16 Azure SQL Database Basic, Standard, Premium Pricing per performance Max 2 GB Web and Business Pricing per Size Max 150 GB Pricing Calculator

17 Azure SQL Database vs SQL Server Authentication Sql Server authentication Windows authentication Sql database schema Use command Log shipping Database mirroring Encryption Connection limitations Auto disconnect TSQL commands subset

18 Azure SQL Database vs SQL Server II Create/ alter (encryption limitations) Procedures, Functions User, Login View Etc. Full text search Global temporary tables Cross database queries Filestream Backup, Restore Transact-SQL debugging

19 SQL Database Scalability Replication Copy entire table / Database Partitioning Split into more databases Clustering Multiple application servers to access same database Sharding Split tables horizontally Join

20 Azure SQL Database - scaling out Horizontal / Vertical Custom Sharding Own mechanism for shards Horizontal Scale Out Shard 1 Shard 2 Shard 3 Shard N Elastic Scale Application define mapping to shards Vertical Scale Out Premium Shard 1 Shard 2 Shard 3 Shard N Basic

21 Azure SQL Database Custom Sharding Tenant End customer for SAAS Single tenant Multi tennant

22 Azure SQL Database Single tenant sharding DB 1 DB 2 DB N Tenant 1 Tenant 2 Tenant N Client application Shard map

23 Azure SQL Database Multi tenant sharding DB 1 DB 2 DB N Tenant 1 Tenant 3 Tenant 2 Tenant N Client application Shard map

24 Azure SQL Database - Elastic scale Shard map management Data dependent routing Determine shard Multi-Shard Queries (MSQ) UNION ALL Split-Merge service Customer hosted service Shrinking & growing data capacity

Microsoft Azure Storage

Microsoft Azure Storage Enabling the Digital Enterprise MICROSOFT AZURE STORAGE (BLOB/TABLE/QUEUE) July 2015 The goal of this white paper is to explore Microsoft Azure Storage, understand how it works