CockroachDB on DC/OS. Ben Darnell, CTO, Cockroach Labs

Transcription

1 CockroachDB on DC/OS Ben Darnell, CTO, Cockroach Labs

2 Agenda A cloud-native database CockroachDB on DC/OS Why CockroachDB Demo!

3 Cloud-Native Database

4 What is Cloud-Native? Horizontally scalable Individual machines aren t special Built to handle failures

5 Why? Adapt to Change Grow and shrink based on demands Easy, rapid deployment Reorganize to reduce load and latency Load follows the sun

6 Traditional Databases More comfortable scaling vertically than horizontally Primaries vs secondaries vs read-only replicas Failover is often manual Asynchronous replication may lose data

7 Cloud-Native CockroachDB Add nodes with automatic rebalancing Any node can serve any role Automatic failover with consistent replication

8 CockroachDB on DC/OS

9 DC/OS Provides an Ideal Environment Elastic allocation and scheduling Discovery, VIPs, and load balancing Weakly persistent local storage

10 Storage Dilemma RAID or network-attached storage is expensive CockroachDB supplies its own redundancy Naive container deployments could lose all local volumes at once DC/OS SDK provides persistent scheduling

11 DC/OS Package dcos package install cockroachdb Starts a 3-node cluster Add more later by changing NODE_COUNT variable Or one-click install through the web

12 Why CockroachDB? An open source SQL database for global cloud services.

13 Why CockroachDB? Distributed SQL Data integrity at global scale Multi-active availability Consistent transactions

14 Distributed SQL A database that grows with your application

15 What is Distributed SQL? SQL with ACID semantics Runs across multiple database servers Distributed storage Distributed compute

16 Distributed Storage Tables scale to any size No manual partitioning necessary Automatic rebalancing as machines are added

17 Distributed Execution Nodes with data compute their portion of results SELECT SUM(duration) FROM sessions

18 Flexible Scheduling Schedule CockroachDB nodes......near your application servers for lower latency...or on your application nodes when you have underutilized disks

19 Global Distribution Span three datacenters to survive major outages Put data close to your customers

20 Multi-Active Availability Survive disasters

21 Disaster Recovery, Version 1.0 DC1 DC2 microservices{ Remote Backup

22 Recovery 2.0: Active/Passive DC1 DC2 async Primary Secondary

23 Recovery 3.0: Active/Active DC1 DC2

24 Multi-Active Availability DC1 DC2 consensus consensus consensus DC3

25 Consistent Transactions Data Integrity at Global Scale

26 Consensus Replication A majority of replicas must agree to commit Failovers never lose committed data Raft consensus algorithm

27 Distributed Transactions Transactions can span rows, tables, even databases ACID semantics Serializable isolation

28 Demo

29

30

31

32

33

34

35

36

37

38 Status

39 Current Version: First production-ready release in May 2017 Distributed SQL Multi-active availability Enterprise: Incremental, distributed backup/restore

40 Very Soon: 1.1 Ruggedization Diagnostic & debugging tools Performance SQL coverage Fast data import (CSV)

41 Up Next Performance SQL feature parity (JSONB) Change data capture (via Kafka) Global data architecture Enterprise: row-level partitioning

42 The End

43 Appendix

44 Target Customers Is CockroachDB right for you?

45 Fastest growing companies in the European Gaming Company world are using CockroachDB to solve their global data challenges $2B+ gaming company that uses CockroachDB to improve performance and availability for their global customer base. $60B+ web services company that uses CockroachDB to power their knowledge graph Real-time gaming platform that uses CockroachDB as the enabling technology to deliver their PaaS Proprietary & Confidential

46 Is CockroachDB right for you? Global, scalable cloud services We hope this eventually means everyone Not for: Latency-sensitive writes Unstructured data Efficient OLAP processing (for now) Analytics in a non-sql language (mapreduce)

47 Correctness or Performance? We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions Authors of Google s Spanner paper

48 CockroachDB Internals Thorax, wings, eyes, legs,...

49 Layered architecture SQL Transactional KV Distribution Replication Storage

50 Data Distribution SQL Transactional KV Distribution Replication Storage

51 Data Distribution Data is a large collection of Key:Value pairs Data split into Ranges of Key:Value pairs Ranges are ~64 MB of data CockroachDB/Bigtable/HBase/Spanner

52 Data Distribution Options for finding data Hashing Order-Preserving

53 Order-preserving data distribution Pro: efficient scans over total ordering of data Con: requires additional indexing

54 Keyspace as Monolithic, Sorted Map

55 Ranges Form the Keyspace

56 Ranges Map to Nodes

57 Data Lookup

58 Consensus SQL Transactional KV Distribution Replication Storage

59 Consensus Replication Replicate to N (where N >= 3) nodes Commit happens when a quorum have written the data CockroachDB/Etcd/Spanner/Aurora/Zookeeper/...

60 Consensus Replication Raft is our consensus protocol of choice. Run one consensus group per range of data

61 Consensus Replication Put cherry Leader apricot banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape

62 Consensus Replication Put cherry Leader apricot Put cherry banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape

63 Consensus Replication Put cherry Leader apricot Put cherry banana blueberry cherry grape Write committed when 2 out of 3 nodes have written data Follower apricot banana blueberry cherry grape Follower apricot banana blueberry grape

64 Consensus Replication Put cherry Leader Put cherry apricot Ack banana blueberry cherry grape Ack Follower apricot banana blueberry Follower apricot banana blueberry cherry grape grape

65 Consensus Replication Put cherry Leader apricot Put cherry banana blueberry cherry grape Ack Follower apricot banana blueberry Follower apricot banana blueberry cherry cherry grape grape

66 Consensus Replication What happens during failover?

67 Consensus Replication: Failover Put cherry Leader apricot banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape

68 Consensus Replication: Failover Put cherry Leader Elect a new Leader apricot banana blueberry cherry grape Follower apricot banana blueberry Follower apricot banana blueberry grape grape

69 Consensus Replication: Failover Follower apricot Leader apricot banana blueberry On failover, only data written to a quorum is considered present Follower apricot banana grape banana blueberry blueberry cherry grape grape

70 Consensus Replication: Failover Read cherry Key not found Follower apricot Leader apricot banana blueberry Follower apricot banana grape banana blueberry blueberry cherry grape grape

71 Consensus Replication Consensus provides atomic writes But only for each range What about operations that hit multiple ranges?

72 Distributed Transactions SQL Transactional KV Distribution Replication Storage

73 Transactions Supports ACID semantics All-or-nothing Serializable [default] & Snapshot isolation levels

74 Distributed Transactions Transaction record Keyed by a transaction UUID Located on first written range Atomic commit or abort via write to txn record One phase commit fast path

75 Distributed Transactions 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) grape

76 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango Txn-1 Record Status: PENDING apricot cherry (txn 1) grape lemon mango (txn 1) orange

77 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango 3. Commit Txn 1 Txn-1 Record Status: COMMITTED apricot cherry (txn 1) grape lemon mango (txn 1) orange

78 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango 3. Commit Txn 1 4. Clean up intents Txn-1 Record Status: COMMITTED apricot cherry grape lemon mango orange

79 Distributed Transactions 1. Begin Txn 1 Put cherry 2. Put mango 3. Commit Txn 1 4. Clean up intents 5. Remove Txn 1 apricot cherry grape lemon mango orange

80 Distributed Transactions That s the happy case What about conflicting transactions?

81 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) grape

82 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry grape

83 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record grape

84 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record Wait while PENDING grape

85 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn1 Txn-1 Record Status: COMMITTED apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record Wait while PENDING grape

86 Distributed Transactions (read conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn1 Txn-1 Record Status: COMMITTED apricot cherry (txn 1) 1. Begin Txn 2 2. Get cherry Check Txn-1 record Wait while PENDING 3. Commit Txn2 grape

87 Distributed Transactions What about write / write conflicts?

88 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING apricot cherry (txn 1) grape

89 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry apricot cherry (txn 1) grape

90 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record apricot cherry (txn 1) grape

91 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry Txn-1 Record Status: PENDING Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record Wait while PENDING apricot cherry (txn 1) grape

92 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn 1 Txn-1 Record Status: COMMITTED Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record Wait while PENDING apricot cherry (txn 1) grape

93 Distributed Transactions (write conflict) 1. Begin Txn 1 Put cherry 2. Commit Txn 1 Txn-1 Record Status: COMMITTED Txn-2 Record Status: PENDING 1. Begin Txn 2 2. Put cherry Check Txn 1 record Resolve cherry (txn 1) Put cherry apricot cherry (txn 2) cherry grape

94 Click to edit title Click to edit text Second level Third level Fourth level» Fifth level