Azure Cosmos DB. Planet Earth Scale, for now. Mike Sr. Consultant, Microsoft

Transcription

1 Azure Cosmos DB Planet Earth Scale, for now Mike Sr. Consultant, Microsoft

2

3 Mission-critical applications for a global userbase need Building globally distributed applications

4 Design Goals Enable customers to elastically scale throughput and storage based on demand, globally. Enable customers to build highly responsive, mission-critical applications. Ensure that the system is always on. Enable developers to write correct globally distributed applications. Offer stringent financially-backed comprehensive SLAs Relieve the developers from the burden of database schema/index management and versioning. Natively support multiple data models and popular APIs for accessing data. Operate at a very low cost... to provide customers best price.

5 Elastically scalable storage Pay per GB used No partition management, no limits Automatically indexed SSD storage Global distribution across Azure regions Automatic expiration via TTL

6 9 PM PST Less throughput Elastically scalable throughput Pay by the hour, only what you need No partition management, no limits Change throughput at any time, effective in seconds More throughput Scale from 100 to 100s of millions of requests/sec across any number of regions 11 PM PST More throughput Now supports requests/min to handle spikes costeffectively Less throughput

7 Provisioned request / sec Black Friday Nov 2016 Dec 2016 Time Hourly throughput (request/sec)

8 Global distribution from the ground-up Worldwide presence Automatic multi-region replication Multi-homing APIs All endpoints are logical, by default Apps don t need to be redeployed during regional failover Designed for HA - Manual and automatic failovers Latency, throughput, consistency, and availability guarantees

9 Guaranteed low latency Read < 1 ms Writes < 6 ms Globally distributed with reads and writes served from local region 50% 99% Read < 10 ms Writes < 15 ms Write optimized, latch-free database engine designed for SSDs Synchronous and automatic indexing at sustained ingestion rates Single-digit millisecond latency at any scale

10 Throughput SLA Consistency SLA Availability SLA Enterprise-grade SLAs Globally distributed database SLAs 1. latency vs. consistency tradeoffs (in steady state) 2. availability vs. consistency tradeoff (during failures) 3. throughput vs. consistency tradeoffs during all times 4. throughput vs. latency tradeoffs during all times 99 th percentile SLA 1 Simply offering high availability SLAs are not sufficient! Cosmos DB: 99.99% HA within a single region % across regions

11 Consistency Level Strong Bounded Staleness Session Consistent Prefix Eventual Guarantees Linearizability Reads lag behind writes by k prefixes or t interval Monotonic reads, monotonic writes, readyour-writes, write-followsreads Updates returned are some prefix of all the updates, with no gaps Eventually consistent Well defined consistency models Global distribution forces us to navigate the CAP theorem Writing correct distributed applications is hard Five well-defined consistency levels Intuitive and practical with clear PACELC tradeoffs Can be overridden on a per-request basis Replicated Data Consistency Explained Through Baseball Tunable data consistency levels in Azure Cosmos DB

12 Guarantee Consistency Performance Availability Strong Excellent Poor Poor Bounded Staleness Good Okay Poor Session Okay Okay Okay Consistent Prefix Okay Good Excellent Eventual Poor Excellent Excellent

13 Azure Cosmos DB 5 well-defined consistency models 01 Bounded Staleness Session Consistent Prefix Clear Tradeoffs Latency Availability Throughput Strong Eventual

14 Multi-model, multi-api First party and popular third-party OSS APIs Key-value, Document, Columnar, and Graph DocumentDB (SQL and JavaScript), MongoDB, Table, Gremlin and Cassandra (preview). Supported across number of programming languages More APIs to be added

15 Database account Core Data Model / Database The core content-model of Cosmos DB s database engine is based on atom-record-sequence (ARS). /dbs/{id} Atoms consist of a small set of primitive types /users/{id} User {Container} /containers/{id} like string, bool, and number. Records are structs composed of these types. Sequences are arrays consisting of atoms, Permission {Item} records, or sequences. /permissions/{id} /items/{id}

16 {Container} Database account Projections / Database /dbs/{id} Collection Table Graph... /users/{id} User /containers/{id} {Container} Permission /permissions/{id} {Item} /items/{id} {Item} Projections Document Row Node Edge...

17 SQL MongoDB Azure Cosmos DB more coming soon Key-Value Column-family Documents Graph Global distribution Elastic scale out Guaranteed low latency Five consistency models Comprehensive SLAs A globally-distributed, multi-model database service

18 Table API Premium experience for Azure Table Storage Secondary indexes, dedicated throughput, low latency, global distribution for Table storage customers Backwards compatible with existing Table SDKs Roadmap: update for standard Tables, optimized for storage Roadmap: seamless migration to Cosmos DB

19 Graph API Social networks, recommendations, logistics, and IoT Globally distributed, horizontally scalable graphs Supports Apache TinkerPop Gremlin Models real world entities and relationships OSS friendly

20 Cassandra API (Preview) Scalable storage size and throughput. Turn-key global distribution Single-digit millisecond latencies at the 99th percentile. Five well-defined consistency levels Automatic indexing of data without requiring you to deal with schema and index management. Guaranteed high availability, all backed by industryleading SLAs

21 How do you chose which API to use? 1. If it is a new application and you re looking for the most control and elasticity SQL API 2. If you re using Mongo, Cassandra a) consider using SQL API for the reasons above. b) Use Mongo or API if not willing to change. 3. If you re working with Graph Use Gremlin 4. If you re an existing Tables customer and you re not willing to make changes and looking for premium tables experience use Table API

22 Planet Earth Azure regions Datacenters Stamps Fault domains Container Cluster Various agents Machine Resource Governor Transport Admission control Database engine Replica Resource Manager Language Runtime(s) Hosts Query Processor RSM Index Manager Bw-tree++/ LLAMA++ Log Manager IO Manager Database engine

23 No schema or index migrations Highly write-optimized database engine Works across every data model Schema Automatic and synchronous indexing Hash, range, and geospatial Fully resource governed Physical index Online and in-situ index transformations

24 Demos

25 Security & Compliance Always encrypted at rest and in motion Highly scalable row level authorization IP firewall rules ISO, EUMC, HIPAA, PCI - certified SOC1, SOC2, FedRAMP, IRS 1075, UK Official audit complete, Q2 2017

26 Cosmos DB: Documentation Azure Code Samples 20 days of Azure Cosmos DB tips Cosmos DB: Excellent Videos Build planet scale mobile apps in minutes (16:17)