NVM Compression: Hybrid Flash- Aware Applica;on Level Compression

Transcription

1 NVM Compression: Hybrid Flash- Aware Applica;on Level Compression Dhananjoy Das Sr. Architect, Advanced Development Group. 5 th October

2 Presenta;on Outline Background Architecture Building blocks EvaluaEon/Benchmarks Summary Q & A ApplicaEon (DB) Name Space/FileSystem Flash TranslaEon Layer NVM- Compression solu;on stack 2

3 ApplicaEon (DB) Background Name Space/FileSystem Flash TranslaEon Layer 3

4 Problem: Exis;ng MySQL ROW Compression 100% 80% 60% 40% 20% 0% 100% Uncompressed TransacEon Rate 80% 20% Row Compressed Compression Performance Penalty 4

5 ROW- Compression Uncompressed data is stored in 16K pages 16K pages are compressed into a fixed compressed page size of 1K, 2K, 4K, 8K Compressed page size is chosen at table creaeon (CREATE TABLE.. KEY_BLOCK_SIZE=8/4/2) Table updates appended to Page ModificaEon Log (mlog) at the end of the compressed (8K) page When mlog gets full, page is recompressed 16K 8K 8K Uncompressed Data Compressed Data Update Compressed Data mlog Insert Insert mlog Insert 5

6 Row- Compression Insert Failures Split Rebalance Recompress Merge + Compress operaeon fails to fit within compressed block size Page is split abempeng to merge contents, triggers an abempt to rebalance the tree 8K Compressed Data >8K Update Insert Insert Compressed Data Insert + Insert Insert 8K Compressed Data mlog 8K Compressed Data mlog 6

7 Row- Compression Architectural Drawbacks Memory Space: Uncompressed & Compressed pages stored in DRAM buffer pool Access: Updates are applied to both copies in memory Capacity cap Fixed compression page size - sets a upper bound on compression Outcome Poor performance on standard benchmarks Poor adopeon (Less than 1% of MySQL users use compression) 7

8 NVM Compression Architecture 8

9 NVM Compression High level approach ApplicaEon operates on sparse address space which is always the size of uncompressed. Compressed data block is wriben in place at same virtual address as the un- compressed. Leaving a hole, empty space in the remainder of space allocated. FTL garbage colleceon naturally coalesces the addresses in physical space, allowing for re- provisioning of physical space. 9

10 NVM Primi;ves Primi;ves Sparse PTRIMs Atomics Details Sparse address space, allows for mapping and allocaeon of blocks on FLASH only as required. Also termed as Thin provisioning. Persistent TRIMs, unlike regular TRIMs which are only hint to garbage- collector, are persistent and enforced across crashes to unmap block that can be reallocated. Atomic- write guarantees that no part of the buffer will be pareally wriben. 10

11 NVM- Compression (page- compression) Only store uncompressed 16KB pages in memory 16K Uncompressed Data Update to storage results in compression. Trim unused 512B sectors in compressed page 16K Compressed Data Unused 16K uncompressed data is now compressed & thin- provisioned down to 3.5K on Flash 16K = (32) 512B Sectors 512B 512B 512B 512B 512B 512B 512B 512B B 512B 512B 512B 512B 512B 512B 512B 3.5K = (7) 512B Sectors on Flash 512B 512B 512B 512B 512B 512B 512B 11

12 Flush opera;on Page Header Data- page Header Compressed Page Header Compressed Page header Index Data ROW Data start end Compress + Add Header Compressed Index Data TRIM unused region Compressed Index Data Garbage/ Unused Garbage/ Unused Footer Page checksum 12

13 NVM- Compression Architectural Benefits Uses natural sparseness of the underlying storage to avoid packing of data Compression is performed only when dirty page is flushed Unused sectors are PTRIM(ed)/unmapped, for reuse Decompression is done on page read, and only uncompressed data is available in buffer- pool pages MulEple compression algorithms are pluggable Modular design and simplified func;onality 13

14 MySQL (DB) Building Blocks NVMFS Flash TranslaEon Layer 14

15 System Interfaces NVM-Compression uses existing system Interfaces System Interface fallocate(offset, len) Func;onality PreallocaEon and extending files/table space fallocate(punch_hole) Unmap/Punch hole operaeon. (issue Persistent TRIMs) io_submit() AIO Transparent Atomic writes 15 15

16 New: Mul;- Threaded Flush Framework 16

17 New: Mul;- Threaded Flush Framework Producer/consumer framework for servicing scalable/distributed operaeon, can be used for purposes other than NVM- Compression Required by NVM- Compression, allows for scalable & efficient servicing of writes CPU#0 CPU#1 CPU#n Flushing Thread 1 2 enqueue wqe (per Page- pool flush) 4 queue report compleeons dequeue request 3 Compress + DIO (per Page) MT- worker threads NVM Compression uses MT- Flush framework to perform compression and PTRIM operaeon in parallel criecal for individual page- flush latency and overall MySQL transaceonal throughput 17

18 NVMFS Non Vola;le Memory FileSystem It s a POSIX compliant filesystem, designed by SanDisk Designed to meet the needs of data intensive workloads Strengths include Direct I/O on large, data intensive workloads (DB workload) Pre- allocaeon of large files is efficient No fragmentaeon/degradaeon with aging of filesystem Re- exports features/primi;ves of the underlying flash 18

19 NVMFS Elimina;ng Duplicate Logic user- space kernel- space Applica;on Linux VFS (virtual file system) abstraceon layer Ext3 file metadata mgmt, block allocaeon, mapping, recycling, ACID updates, logging/journaling, crash- recovery Kernel block layer NVMFS file metadata mgmt PrimiEve Interfaces NaEve Flash TranslaEon Layer block allocaeon, mapping, recycling ACID updates, logging/journaling, crash- recovery 19

20 NVM Primi;ves MySQL already uses Transparent Atomics 20

21 Atomic Writes (no double write) Tradi;onal MySQL Writes Database Server Page A B Buffer DRAM Buffer Page Page A SSD (or HDD) Page B Page A Page C Page B 2 Writes Page C Page A Page B Page C Page C Database Updates to pages A, B, C Copied to memory buffer. Writes to double- write buffer Once acknowledged, commit to the database. MySQL with Atomic Writes Database Server DRAM Buffer 1 Atomic Write Page A iomemory Page B Page A Page C Page B Page A Page B Page C Page C Database Updates to pages A, B, C Copied to memory buffer. MySQL writes ONCE to database, commisng the transaceon with inherent atomicity through Atomic Writes API 21

22 Benchmarks 22

23 Benchmarks Used LinkBench Percona tool tpcc- mysql (TPC- c like) Other Micro- benchmarks 23

24 Storage Savings % improvement LinkBench 10x workload Uncompressed vs. Table with ROW, NVM Compression 60.0% 58.0% 56.0% 54.0% 52.0% 50.0% 48.0% 46.0% 44.0% Row- comp Page- comp Delta- Uncomp% 49.0% 58.5% NVM Compression supports muleple pluggable compression methods including lz77, lz4, lzo * The data show is using zlib(lz77), storage efficiency with lz4 are comparable delta ~3% Cau;on: ResulEng compression raeo depends on Entropy of data along with the algorithm used 24

25 LinkBench 99% OP latencies LinkBench 10x Mean OP latency (msec) Uncomp/NVM- comp/row- comp NVM Compression transaceon latency is 2x- 5x beber compared to ROW Compression and is closed to Uncompressed ADD_NODE UPDATE_NODE DELETE_NODE GET_NODE ADD_LINK DELETE_LINK UPDATE_LINK COUNT_LINK Uncompressed NVM- Comp ROW- Comp 25

26 Benchmark: LinkBench MariaDB /InnoDB 10x Workload (110GB, 50GB- buf- pool) Uncomp vs. Page- Comp vs. Row- Comp NVM Compression transaceon throughput is 2.25x beber than ROW Compression and is <5% of Uncompressed Uncomp Page- Comp Row- Comp 26

27 Scalability with Cores LinkBench 10x workload (110GB, 50GB- buf- pool) Uncomp vs. Page- Comp vs. Row- Comp 24- GHz 32- GHz K K NVM Compression transaceon throughput scales well with the core count (MT- Flush) 42.7K K K K Uncomp Page- Comp Row- Comp 27

28 Benchmark: (TPCC- like) NVM- Compression TPC- C like workload MariaDB warehouses - 75GB DRAM New Order TX MySQL uncompressed MySQL compression Fusion- io NVM Compression 0 Time Time NVM Compression performance is 7x beber than ROW Compression and close to Uncompressed workload perf 28

29 NVMFS Performance - TRIM Handling 8 Micro- benchmark Trim aver write 16 KiB Direct Write + 4KiB TRIM Applica;ons MySQL Page- compression Elapsed Time (normalized) PTRIMs help with efficient garbage colleceon, reallocaeon of unused blocks and reduces WA 1 0 nvmfs xfs 29

30 Flash Endurance Power/Cooling 30

31 Fewer Writes/OP LinkBench 10x workload 50GB buf- pool Uncompressed vs. NVM- Compression *Lower is beber 31

32 4x Bejer Flash Endurance Compression 2x fewer writes to flash Atomics 2x fewer writes by disabling double- writes Persistent TRIMs Lower write amplificaeon benefits for flash Usable for flash with fewer write cycles, like TLC/3BPC 32

33 Power/Cooling LinkBench 10x workload 50GB buf- pool Uncompressed vs. NVM- Compression 100% 90% 80% % peak uncompressed 70% 60% 50% 40% 30% Fewer writes to Flash with NVM Compression improves savings on power usage and cooling PG- Wabs Uncmp- Wab Delta- Temp 20% 10% 0% Time 33

34 Summary NVM- Compression Design combines applicaeon level compression with flash awareness Compression is performed by applicaeon at the point of page- flush Leverages FTL primieves for block management and garbage colleceon System standard interfaces (PTRIMs/Atomic- write) Pluggable compression libraries Benchmark OLTP workload results Storage saving 2x+ Scalable & superior performance 34

35 Lessons learnt Reducing writes can have dramaec performance benefit even if it comes at the cost of CPU based compression. CapabiliEes embedded in flash FTLs, like high performance GC, can be used to replace complex ApplicaEon level block management very effecevely. File system support is required to maintain manageability. File systems passing and implemeneng primieves efficiently is criecal for performance. 35

36 Q & A 36

37 Thank You 37

38 Backup Slides 38

39 Announcements Early access NVM- Compression solueon Early access NVMFS Release by community Vendor Release Download SkySQL MariaDB hbp://bazaar.launchpad.net/~jplindst/maria/10.0- FusionIO Percona Percona Server 5.6 hbp://code.launchpad.net/~gl- az/percona- server/5.6- pagecomp_mxlush Oracle MySQL DMR (Atomics) hbp://dev.mysql.com MySQL Labs release (Compression) hbp://labs.mysql.com/ 39

40 Read Index- Data- BLOCK (16K) Page Header Data- page Header Compressed Page header Index Data ROW Data start end DeCompress + Fill User Buffer Compressed Index Data Garbage/ Unused UnMapped Region (zero filled) Footer Page checksum 40

41 How to NVM Compress 41

42 How to NVM- Compress Setup MySQL (/etc/my.cnf) Create Table opeons Compression staesecs 42

43 MySQL Seong /etc/my.cnf I/O engines supported Innodb XtraDB Compression Protocols supported LZ4 LZ77 LZMO 43

44 Create Table Sample # 1 (MariaDB ) Reserved Keywords to use for create table opeons PAGE_COMPRESSION PAGE_COMPRESSION_LEVEL Enable PAGE_COMPRESSION 44

45 Create Table sample #2 (MariaDB ) Reserved Keywords to use for create table opeons DIRECTORY ATOMICS_WRITES PAGE_COMPRESS op;ons.. Create Table under specified directory Enable Page- compression + Atomic writes 45

46 NVM Compression: Sta;s;cs Show status like Innodb_page_compression_% Show status like Innodb_n% Variable_name Value Innodb_page_compression_saved Innodb_page_compression_trim_sect Innodb_page_compression_trim_sect Variable_name Value Innodb_num_index_pages_written Innodb_num_pages_page_compressed Innodb_num_page_compressed_trim_op Innodb_num_page_compressed_trim_op_saved Innodb_num_pages_page_decompressed

47 LinkBench Seong (1x vs. 10x) 1x workload Bump up the workload from default 1x (10Mil) to 10x using maxid Change runeme, using max;me 47

48 OK What Happened with Latency? 48