A Light-weight Compaction Tree to Reduce I/O Amplification toward Efficient Key-Value Stores

Transcription

1 A Light-weight Compaction Tree to Reduce I/O Amplification toward Efficient Key-Value Stores T i n g Y a o 1, J i g u a n g W a n 1, P i n g H u a n g 2, X u b i n He 2, Q i n g x i n G u i 1, F e i W u 1, a n d C h a n g s h e n g Xie 1 1 W u h a n N a t i o n a l L a b o r a t o r y f o r O p t o e l e c t r o n i c s H u a z h o n g U n i v e r s i t y o f S c i e n c e a n d T e c h n o l o g y 2 T e m p l e U n i v e r s i t y

2 Background Outline LWC-tree (Light-Weight Compaction tree) LWC-store on SMR drives Experiment Result Summary

3 Background Key-value stores are widespread in modern data centers. Better service quality Responsive user experience The log-structured merge tree (LSMtree) is widely deployed. RocksDB, Cassandra, Hbase, PNUTs, and LevelDB Memory Disk L0 L1 L2 Ln compaction Write (key, value) 1 Log High throughput write SSTable MemTable Immutable MemTable Fast read performance

4 Background LSM-tree Memory a-c a b c sort 2 Disk L 0 1 read write 3 Above 10x L 1 a-c L 2 a b c L n The overall read and write data size for a compaction: 8 tables 13x This serious I/O amplifications of compactions motivate our design!

5 Background LWC-tree (Light-Weight Compaction tree) LWC-store on SMR drives Experiment Result Summary

6 LWC-tree Aim Alleviate the I/O amplification Achieve high write throughput No sacrifice to read performance How Keep the basic component of LSM-tree Keep tables sorted Keep the multilevel structure Light-weight compaction reduce I/O amplification Metadata aggregation reduce random read in a compaction New table structure, DTable improve lookup efficient within a table Workload balance keep the balance of LWC-tree

7 LWC-tree Light-weight compaction Aim Reduce I/O amplification How append the data and only merge the metadata Read the victim table Sort and divide the data, merge the metadata Overwrite and append the segment Memory Disk L 0 L 1 L 2 L n 1 read a-c a-c sort a b c a b c Overwrite and append 3 Reduce 10 x amplification theoretically (AF=10) The overall read and write data size for a light-weight compaction: 2 tables. (In LSM-tree, the overall data size for a conventional compaction: 8 tables.)

8 LWC-tree Metadata Aggregation Aim Reduce random read in a compaction Efficiently obtain the metadata form overlapped Dtables Li a-c How Cluster the metadata of overlapped DTables to its corresponding victim Dtable after each compaction Li+1 a b c a b c A light-weight compaction

9 LWC-tree Metadata Aggregation Aim Reduce random read in a compaction Efficiently obtain the metadata form overlapped Dtables Li a-c How Cluster the metadata of overlapped DTables to its corresponding victim Dtable after each compaction Li+1 a a b b c c Metadata aggregation after light-weight compaction

10 LWC-Tree DTable Dtable Index block Overlapped Dtables Metadata Magic data Origin data Segment 1 (append data) Segment 2 (append data) Metadata Origin index Data_block i Data_block i+1 Segment 1 Index Filter blocks Overlapped Meta_index block Meta_index block Index block footer Segment 2 Index Aim Support Light-weight compaction Keep the lookup efficiency within a DTable How Store the metadata of its corresponding overlapped Dtables Manage the data and block index in segment Index block in segment

11 LWC-Tree Workload Balance Aim Keep the balance of LWC-tree Improve the operation efficiency How Deliver the key range of the overly-full table to its siblings after light-weight compaction Data volume Data block n DTable number in Level L i Range adjustment Advantage no data movement and no extra overhead Li Li+1 a-c d a b c d a b c-d a-b c d Light-weight compaction

12 Background LWC-tree (Light-Weight Compaction tree) LWC-store on SMR drives Experiment Result Summary

13 LevelDB on SMR Drives SMR(Shingled Magnetic Recording) Overlapped tracks Band & Guard Regions Random write constrain LevelDB on SMR Multiplicative I/O amplification Figure from Fast 2015 Skylight A Window on Shingled Disk Operation

14 LevelDB on SMR Drives SMR(Shingled Magnetic Recording) Overlapped tracks Band & Guard Regions Random write constrain LevelDB on SMR Multiplicative I/O amplification Band size 40 MB WA(Write amplification of LevelDB) 9.83x AWA (Auxiliary write amplification of SMR) 5.39x MWA (Multiplicative write amplification of LevelDB on SMR ) 52.58x Amplification RAtio WA MWA band size (MB) This auxiliary I/O amplifications of SMR motivate our implementation!

15 LWC-store on SMR drive Aim Eliminate the auxiliary I/O amplification of SMR Improve the overall performance How A DTable is mapped to a band in SMR drive Segment appends to the band and overlaps the out-of-date metadata Equal division: Divide the DTable overflows a band into several subtables in the same level

16 Background LWC-tree (Light-Weight Compaction tree) LWC-store on SMR drives Experiment Result Summary

17 Configuration Test Machine SMR Drive CMR HDD SSD Experiment Perimeter 16 Intel(R) Xeon(R) CPU E GHz processors Seagate ST5000AS0011 Seagate ST1000DM003 Intel P LevelDB on HDDs (LDB-hdd) 2. LevelDB on SMR drives (LDB-smr) 3. SMRDB* An SMR drive optimized key-value store reduce the LSM-tree levels to only two levels (i.e., L0 and L1) the key range of the tables at the same level overlapped match the SSTable size with the band size 4. LWC-store on SMR drives (LWC-smr)

18 Experiment Load (100GB data) Random load Large amount of compactions LWC-store 9.80x better than LDB-SMR LWC-store 4.67x better than LDB-HDD LWC-store 5.76x better than SMRDB Rnd.load throughput (MB/s) Random put 4.67x LDB-SMR LDB-HDD SMRDB LWC-SMR Sequential load No compaction Similar Sequential load performance Seq.load throughput (MB/s) Sequential put LDB-SMR LDB-HDD SMRDB LWC-SMR

19 Experiment read (100K entries) Look-up 100K KV entries against a 100GB random load database Seq.read throughput (MB/s) Sequential get LDB-SMR LDB-HDD SMRDB LWC-SMR Rnd.read latency (ms) Random get LDB-SMR LDB-HDD SMRDB LWC-SMR

20 Experiment compaction (randomly load 40GB data) Compaction performance in microscope LevelDB: number of compactions is large SMRDB: data size of each compaction is large LWC-tree: small number of compactions and small data size Overall compaction time LWC-smr gets the highest efficiency

21 Experiment compaction (randomly load 40GB data) Compaction performance in microscope LevelDB: number of compactions is large SMRDB: data size of each compaction is large LWC-tree: small number of compactions and small data size Overall comp time (s) Overall compaction time LWC-smr gets the highest efficiency 0 LDB-SMR LDB-HDD SMRDB LWC-SMR Overall compaction time

22 Experiment Write amplification Competitors LWC-SMR LDB-SMR Write amplification (WA) Write amplification of KV-store Auxiliary write amplification (ARA) Auxiliary write amplification of SMR multiplicative write amplification (MWA) Multiplicative write amplification of KV stores on SMR Write amplification Multiplicative write amplification LWC-SMR.WA LDB-SMR.WA LWC-SMR.AWA LDB-SMR.AWA LWC-SMR.MWA LDB-SMR.MWA MB 30MB 40MB 50MB 60MB Band Size 20MB 30MB 40MB 50MB 60MB Band Size x

23 Experiment LWC-store on HDD and SSD

24 Background LWC-tree (Light-Weight Compaction tree) LWC-store on SMR drives Experiment Result Summary

25 Summary LWC-tree: A variant of LSM-tree Light-weight compaction Significantly reduce the I/O amplification of compaction LWC-store on SMR drive Data management in SMR drive eliminate the auxiliary I/O amplification from SMR drive Experiment result high compaction efficiency high write efficiency Fast read performance same as LSM-tree Wide applicability

26 Thank you! QUESTIONS?