DHRUBA BORTHAKUR, ROCKSET PRESENTED AT PERCONA-LIVE, APRIL 2017 ROCKSDB CLOUD

Size: px

Start display at page:

Download "DHRUBA BORTHAKUR, ROCKSET PRESENTED AT PERCONA-LIVE, APRIL 2017 ROCKSDB CLOUD"

Lora Lambert
6 years ago
Views:

1 DHRUBA BORTHAKUR, ROCKSET PRESENTED AT PERCONA-LIVE, APRIL 2017 ROCKSDB CLOUD

2 WHAT ARE WE TALKING ABOUT? OUTLINE Why RocksDB-Cloud? Differences from RocksDB Goals, Design, Architecture Next Steps

3 OUR INHERITANCE ROCKSDB STORAGE ENGINE Open Sourced from Facebook Engineering Log Structured MergeTree Embedded c++/java/go library Available as MyRocks and MongoRocks Used at Microsoft, Yahoo, Netflix,

4 OUR INHERITANCE ROCKSDB FOCUS ON EFFICIENCY MyRocks on FB workload DB Size (GB) 50% smaller InnoDB RocksDB

5 OUR INHERITANCE ROCKSDB FOCUS ON EFFICIENCY MyRocks on FB workload 50% smaller 50% lesser IOs GB Written InnoDB RocksDB 0 Day MyRocks is supported by Percona

6 WHY ROCKSDB-CLOUD? HAVE YOU USED ROCKSDB FOR CLOUD APPS? Rockset started to use RocksDB on AWS build our own replication engine build our own backup system custom code for hot/cold placement Shared Nothing is dead. 2017, nedbday/2017/talks/ dewitt.pptx RAM, NVMe, SSD, disk

7 WHY ROCKSDB-CLOUD? ROCKSDB-CLOUD IS CHEAPER RocksDB-Cloud uses locally attached SSD and S3 3x cheaper than 3 way replication If I redesigned HDFS today It won't use 3 way replication n times cheaper than EBS, n > 1

8 WHY ROCKSDB-CLOUD? VISION FOR ROCKSDB-CLOUD Optimized for Cloud Applications Support AWS, Google Storage, Azure Pluggability for other cloud vendors

9 ROCKSDB-CLOUD GOALS Durability of data inspite of machine failures Replication of data across machines

10 ROCKSDB-CLOUD GOALS Durability of data inspite of machine failures Replication of data across machines Auto placement of hot/cold data on cloud storage hierarchy Copyright Computer History Museum

11 ROCKSDB-CLOUD GOALS Durability of data inspite of machine failures Replication of data across machines Auto placement of hot/cold data on cloud storage hierarchy Portability across cloud vendors

12 ARCHITECTURE 1: APPLICATION LOG IN FRONT OF DATABASE tail data from distributed log storage writes Cloud Application RocksDB-Cloud reads queries memtable cache block cache flush sst file to local SSD flush sst file to cloud storage persistent read cache on SSD Cloud Storage

13 ARCHITECTURE 1 TAILING A LOG Tail data from Kafka topic into RocksDB-Cloud Open (local_directory, S3 bucket name) SST file copied to S3 at the time of file close keep or delete local set file Every change to MANIFEST is copied to S3 Kafka state stored in RocksDB-Cloud

14 ARCHITECTURE 1 TAILING A LOG Recover data when machine fails Open (local_directory, S3 bucket name) MANIFEST downloaded from S3 Download data from sst files on demand Local SSD/disk as persistent cache Restart tailing from Kafka

15 ARCHITECTURE 1: ZERO COPY CLONES tail data from distributed log storage Server RocksDB-Cloud queries write read Cloud Bucket A tail data from distributed log storage read Cloned Server RocksDB-Cloud write read Cloud Bucket B queries served by either server Instantaneous clone creation Both machines run at their own speeds True master-master configuration

16 ARCHITECTURE 1: MASTERLESS REPLICATION ZERO COPY CLONES Purger runs on every Machine Deletes sst files that are not part of any clones Both machines run at their own speeds True master-master configuration

17 ARCHITECTURE 2: DATABASE WAL ENABLED Cloud Log Storage, Kinesis WAL flush sst file to local SSD writes memtable cache Cloud Application RocksDB-Cloud flush sst file to cloud storage reads block cache persistent read cache on SSD updates queries Cloud Storage

18 ARCHITECTURE 2: ZERO COPY CLONES WAL writes Server RocksDB-Cloud updates queries write read Cloud Log Storage, Kinesis WAL tailer Cloud Bucket A read Cloned Server RocksDB-Cloud queries write read Clones are read-only replicas Cloud Bucket B

19 ARCHITECTURE 2 READ WRITE DB Master machine: read-write write WAL on AWS- Kinesis Slave machine: read-only tail Kinesis and apply

20 HIERARCHICAL STORAGE AUTO PLACEMENT OF HOT/COLD DATA All Levels L0 - Ln reside in S3 Levels L0 - L2 typically reside in local SSD and S3 Cache data from S3 for reads: persistent cache on locally attached SSD Support for Intel NVMe

21 PORTABILITY ACROSS CLOUD VENDORS SEAMLESS COPY AMONG S3, AZURE, GOOGLE App on Azure can access AWS S3 Storage App on Google Cloud can access Azure Storage same API on all cloud platforms write write RocksDB Cloud App AWS S3 RocksDB Cloud App read read Azure Google Cloud

22 LOW ADOPTION COST COMPATIBILITY Pure Open Source API compatible with stock RocksDB Data format compatible with stock RocksDB License compatible with stock RocksDB BSD License

23 FUTURE WORK NEXT STEPS Support for large size objects Support encryption-at-rest

24 OPEN SOURCE COMMUNITY COLLABORATORS

25 COME HACK WITH US REFERENCES Source code: Dev discussions: Slack Channel:

RocksDB Embedded Key-Value Store for Flash and RAM

RocksDB Embedded Key-Value Store for Flash and RAM Dhruba Borthakur February 2018. Presented at Dropbox Dhruba Borthakur: Who Am I? University of Wisconsin Madison Alumni Developer of AFS: Andrew File