How To Rock with MyRocks. Vadim Tkachenko CTO, Percona Webinar, Jan

Transcription

1 How To Rock with MyRocks Vadim Tkachenko CTO, Percona Webinar, Jan

2 Agenda MyRocks intro and internals MyRocks limitations Benchmarks: When to choose MyRocks over InnoDB Tuning for the best results

3 LSM-trees writes merge merge merge merge memory disk Write-optimized data structure

4 Write-optimized? Write immediately Do the heavy work sometime later writes Memtable (in memory) Flush when full L0 Key=>value WAL (redo-log) memory storage

5 Traditional engines (B+ tree) Write may trigger read Flush in background writes cache (in memory) Data File Read if updated data is not in memory WAL (redo-log) memory storage

6 B+tree performance

7 Write-optimized problems Reads Do the heavy work sometime later è may cause multiple copies in multiple files Reads in general slower than in B+ Tree engines

8 Write-optimized problems Reads Unique keys è force read for constraint check Writes to unique keys Memtable (in memory) Flush when full L0 Foreign keys the same (not supported at the moment) Constrain check Read if updated data is not in memory Block cache WAL (redo-log) memory storage

9 Background jobs Sorted run of key=>value pairs, partial data sets Final data set, sorted by (key) Flush when full L0 L1 L6 Memtable (in memory) Compact (merge) when full memory storage Compact (merge) when full Compact (merge) when full

10 Space amplification L1 L2 L3 L6 Final data set N GB storage N*10 GB N*100 GB N* GB

11 Space amplification If N=1 and the final size is GB (100TB), then extra size: = GB amplification is 11.1%

12 Space amplification What if the final size is not 100TB? The space amplification will be bigger than 11% Dynamic compaction N is calculated dynamically as we change data level_compaction_dynamic_level_bytes = true (now default in 8.0) Dynamic compaction allows to keep space amplification minimal

13 Space amplification in B+ tree Depends on insertion order Worst case all pages split in half Space amplification: 0% if insert in sequential order up to 50% if random (the worst case scenario) The real number is somewhere in the middle

14 Slow reads read amplification in LSM Brief description of section contents

15 Read amplification - Slow reads in MyRocks L0 L1 L6 Point Read (select where key=n) Reads are slow. This is what we pay for not doing reads at the time of write Binary search on the whole level

16 Slow reads Point Read (select where key=n) L0 L1 L6 Block cache (memory) rocksdb_block_cache_size Binary search on the whole level

17 Slow reads Quick answer if data does not exist L0 L1 L6 Point Read (select where key=n) bloom bloom Trick: Bloom filters Stored in memory Allow quickly dismiss levels that do not have data. Enabled by filter_policy=bloomfilter:10:false bloom

18 Slow reads L0 L1 L6 Point Read (select where key=n) bloom bloom L6 bloom filter may be not needed if the most queries ask data that exists bloom SELECT name FROM users WHERE

19 Slow RANGE reads L0 L1 L6 Range Read (select where key > M and key < N) Range scan Bloom filters DO NOT WORK on range. Range queries are going to be slow. There is a hope: Succinct Range Filter (SuRF) in the research stage by Carnegie Mellon University Range scan

20 Cost of reads Thanks Mark Callaghan for making the math Assume 8 bln rows table Point lookup B+ tree: ~33 comparison operations LSM tree: ~80 comparison operations

21 So If reads are slow is MyRocks useful? Let s not forget the bigger picture. To select by a non-primary key we need indexes. The more indexes - the slower writes in B+ tree

22 Overall performance Totally unscientific chart just for the illustration InnoDB (B+ tree) Many variables are in play: Memory size Storage performance Datasize Workload MyRocks (LSM Tree) Size of indexes / data If you are here InnoDB is better If you are here MyRocks is better

23 Where do I see a fit for MyRocks? Relatively big datasets Over ~100GB in size 5GB is not big With multiple indexes Write-intensive workloads

24 MyRocks how to get it Brief description of section contents

25 The Source of truth RocksDB key-value library Only in source code MyRocks engine for MySQL Implements MySQL api and calls RocksDB Only in source code Only for MySQL 5.6 You need to know what RocksDB version it works with

26 Production packages Percona Server 5.7 and 8.0 All heavy-integration work and testing

27 Things to know about MyRocks All files are in a single.rocksdb directory No separation per-database or per-table sst sst sst sst sst sst LOG file contains a lot of useful information. A LOT

28 Default compression LZ4 In Percona Server, all levels are LZ4 compressed by default Zstd is available. Different compression for L6 is possible: compression=klz4compression;bottommost_compression=kzstd Global setting for ALL databases and tables Column families allow settings per table and per index

29 Traditional zlib is slow Great speed/ratio balance Best speed

30 Isolation levels MyRocks supports READ-COMMITTED REPETABLE-READ * MyRocks satisfies ANSI definition of REPETABLE-READ InnoDB applies gap-locking in REPEATABLE-READ MyRocks REPETABLE-READ!= InnoDB REPEATABLE-READ MyRocks Percona Server Error on a statement that requires gap-locking in REPETABLE-READ To keep applications informed that gap-locking is not supported and the result may be different

31 Bulk load Bulk load is problematic Big INSERT SELECT Big ALTER TABLE Big LOAD DATA Better if fits into memory Use rocksdb_bulk_load=1 Also rocksdb_commit_in_the_middle may help if not enough memory Divides a big transactions into smaller Further improvements are coming

32 Backups There is no a good hot-backup solution for Percona Server, yet You can try /scripts/myrocks_hotbackup The integration with Percona XtraBackup is coming in 2019

33 Benchmarks Brief description of section contents

34 The world most popular benchmark SELECT count(*) FROM table Though tempted - Don t use it! Don t use it in general Don t use for MyRocks especially 2 nd most popular SELECT value FROM table WHERE primary_key_id=<random> Still no good

35 To understand MyRocks performance we need the proper mix of datasize, indexes, read/write queries Overall performance InnoDB (B+ tree) MyRocks (LSM Tree) Size of indexes / data

36 Parameters Sysbench-tpcc W (100GB dataset) or 5000W (500GB dataset) Memory setting varies from small to big Innodb_buffer_pool_size + DIRECT_IO rocksdb_block_cache_size + DIRECT_IO Binary logs enabled Without them - bug in transaction coordinator makes commit very slow if multiple engines are enabled We always use binary logs in production anyway

37 Parameters - cont Engines tuned for the full ACID compliance READ-COMMITTED isolation level MyRocks won t accept SELECT FOR UPDATE in REPEATABLE-READ

38 Small % of data cached Data fits into memory

39 MyRocks jitter from Background compaction

40

41

42

43

44 Benchmarks in Cloud (AWS) Brief description of section contents

45 Parameters Amazon volumes io1 provisioned IOPS, the more IOPS the more expensive - IOPS from 1000 to gp2 general purpose, scales with size GB 3300 IOPS GB IOPS - Usually less expensive than IO1 Sysbench 5000W, 500GB Cache size 27GB and 54GB MyRocks uses LZ4 compression, data is highly compressible MyRocks size 100GB

46 For the same throughput MyRocks needs 5000 IOPS InnoDB needs IOPS

47 Cloud efficient MyRocks 5000 IOPS volume InnoDB IOPS volume MyRocks needs less memory resources MyRocks Less IOPS Less memory Smaller storage (compression) è cheaper to run in Cloud

48 MyRocks jitter from Background compaction

49 Scaling IOPS

50 I n InnoDB IO throughput Is limited by instance bandwidth

51 MyRocks parameters rocksdb_block_cache_size The memory to cache data blocks DirectIO (bypass OS cache) or buffered IO? If DirectIO set rocksdb_block_cache_size as big as possible (60-80%) If buffered set 25-40% Compressed data will be cached in OS cache To set DirectIO: rocksdb_use_direct_reads=on rocksdb_use_direct_io_for_flush_and_compaction=on

52 MyRocks parameters rocksdb_max_background_jobs=<num_cpu_cores/4> rocksdb_max_total_wal_size=4g rocksdb_block_size=16384 Better compression comparing to default 4096

53 MyRocks parameters Column family settings For default column family set in rocksdb_default_cf_options Compaction_pri set to kminoverlappingratio (default in PS 8.0) To understand this more: Towards Accurate and Fast Evaluation of Multi-Stage Log- Structured Designs Dynamic compaction: level_compaction_dynamic_level_bytes=true (default in PS 8.0) For better space amplification, see previous slides Bloom filter for all, but the bottom level (default in PS 8.0) block_based_table_factory={filter_policy=bloomfilter:10:false};optimize_filters_for_hits =true;

54 Column families Each column family has: Individual memtable (64MB memory size by default, plan memory accordingly) Bloom filters SST files Column family settings CREATE TABLE t1 (a INT, b INT, PRIMARY KEY(a) COMMENT 'cfname=cf1, KEY kb(b) COMMENT 'cfname=cf2 ) rocksdb_override_cf_options='cf1={compression=knocompression}; cf2={compression=kzstd}'

55 Thank You! Questions?