Azure-persistence MARTIN MUDRA

Azure-persistence MARTIN MUDRA

Storage service access Blobs Queues Tables

Storage service Horizontally scalable Zone Redundancy Accounts Based on Uri Pricing Calculator

Azure table storage Storage Account Account Table Cars Customers Entities Name= Color= Name= Color= Name= Surname=

Azure table storage II PartitionKey Load balancing across storage nodes Partition Range of entities with same key Up to 1KB RowKey Primary key PartitionKey + RowKey Up to 1KB TimeStamp DateTime Ignored on insert & update Optimistic concurrency Supported Types Binary byte[] Boolean bool DateTime DateTime Double double Guid Guid Int32 int Int64 long String string

Azure tables - Query Select Max 1000 items Max 5 sec Pagination Top Filter Max 15 discrete comparisons Output formats Atom Json v 3.0 Xml Supported filters Equal eq GreaterThan gt GreaterThanOrEqual ge LessThan lt LessThanOrEqual le NotEqual ne Supported Boolean operators And Not Or https://myaccount.table.core.windows.net/customers()?$filter=customersince%20eq%20datetime'2008-07-10t00:00:00z'

Azure tables - transaction Batch operations Insert Update Delete OData protocol Same PartitionKey Max 100 operations

Azure tables - scalability Partition distribution across servers Health of the servers Traffic Load balancing Target 500 entities per second / partition DFS layer IO load Server busy / timeout error Retry strategy No retry Fixed backoff Exponential backoff

Azure tables scalability II PartitionKey Granularity Partition Size Advantages Disadvantages Single value Small number of entities Batch transactions are possible with any entity All entities are local and served from the same storage node Single value Large number of entities Entity Group Transactions may be possible with any entity. Multiple values There are multiple partitions Partition sizes depend on entity distribution Batch transactions are possible on some entities Dynamic partitioning is possible Single-request queries possible (no continuation tokens) Load balancing across more partition servers possible Unique values There are many small partitions. The table is highly scalable Range partitions may improve the performance of cross-partition range queries Scaling is limited. Throughput is limited to the performance of a single server. A highly uneven distribution of entities across partitions may limit the performance of the larger and more active partitions Queries that involve ranges may require visits to more than one server. Batch transactions are not possible. Append or prepend-only patterns can affect insert-throughput

Azure tables Scalability performance Query Type PartitionKey Match RowKey Match Performance Rating Point Exact Exact Best Row range scan Exact Partial Better with smaller-sized partitions Bad with very large-sized partitions Partition range scan Partial Partial Good with a small number of partition servers being touched Worse with more servers being touched Full table scan Partial, none Partial, none Worse with a subset of partitions being scanned Worst with all partitions being scanned

Azure blob Storage Account Container Blob Account Pictures IMG001.jpg IMG002.jpg Videos Video001.mpeg

Azure blob - Blob types Block blob Blocks Upload large Parallel upload Commit all at once Page blob Azure Premium storage Pages Commit by page Random reads/writes Ideal for VHD

Azure blob - Block blob Block Up to 4MB 50000 blocks BlockId Base64 string Insert Delete Modify One-week to commit blocks Up to 100000 uncommitted blocks (max 400GB) Transaction 64 MB in max size in single request Commit by block BlockBlob 1 2 3 5 6 7 N

Azure blob Page blob Page 512 bytes Max size 1 TB Write Change Range & offset align to 512 bytes Writes in-place Pageblob 1 2 3 5 6 7 N Commit immediately (by page)

SQL Server Existing application Disaster recovery Full admin rights On-premise resources Active Directory Eliminates hardware Hybrid cloud Pricing Calculator

Azure SQL Database Basic, Standard, Premium Pricing per performance Max 2 GB Web and Business Pricing per Size Max 150 GB Pricing Calculator

Azure SQL Database vs SQL Server Authentication Sql Server authentication Windows authentication Sql database schema Use command Log shipping Database mirroring Encryption Connection limitations Auto disconnect TSQL commands subset

Azure SQL Database vs SQL Server II Create/ alter (encryption limitations) Procedures, Functions User, Login View Etc. Full text search Global temporary tables Cross database queries Filestream Backup, Restore Transact-SQL debugging

SQL Database Scalability Replication Copy entire table / Database Partitioning Split into more databases Clustering Multiple application servers to access same database Sharding Split tables horizontally Join

Azure SQL Database - scaling out Horizontal / Vertical Custom Sharding Own mechanism for shards Horizontal Scale Out Shard 1 Shard 2 Shard 3 Shard N Elastic Scale Application define mapping to shards Vertical Scale Out Premium Shard 1 Shard 2 Shard 3 Shard N Basic

Azure SQL Database Custom Sharding Tenant End customer for SAAS Single tenant Multi tennant

Azure SQL Database Single tenant sharding DB 1 DB 2 DB N Tenant 1 Tenant 2 Tenant N Client application Shard map

Azure SQL Database Multi tenant sharding DB 1 DB 2 DB N Tenant 1 Tenant 3 Tenant 2 Tenant N Client application Shard map

Azure SQL Database - Elastic scale Shard map management Data dependent routing Determine shard Multi-Shard Queries (MSQ) UNION ALL Split-Merge service Customer hosted service Shrinking & growing data capacity