Compressing Relations and Indexes Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft 1 Database Talk Speaker : Jonathan Goldstein Microsoft Researc

Transcription

1 Compressing Relations and Indexes Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft 1 Database Talk Speaker : Jonathan Goldstein Microsoft Research

2 OVERVIEW ffl Motivation ffl Compressing relations with fixed size atributes ffl Page level compression ffl File level compression ffl Experimental results ffl Compressing indexes ffl Compressing index pages ffl Experimental results ffl Related and Future Work ffl Conclusions 2 Database Talk

3 MOTIVATION Benefits: ffl Improved storage requirements. ffl Improved information throughput from disks ffl Potentially improved buffer utilization ffl Improved fanout for indexing structures. 3 Database Talk

4 COMPRESSING RELATIONS(Page Level) Frame of reference (page level) compression: Given the points: f(511,1001),(517,1007),(514,1031)g subtract (511, 1001) from all tuples and the set in binary becomes: f(000, 00000), (110, 00110), (011, 11110)g Only 3 and 5 bits are needed to store these tuples provided that we know (511,1007) was subtracted from them! 4 Database Talk

5 COMPRESSING RELATIONS(Page Level) This compression technique can be applied to fixed length data such that: ffl Decompression can be done on a per field per tuple granularity ffl Pages can stay in memory compressed. 5 Database Talk

6 6 Database Talk COMPRESSING RELATIONS(File Level) How do we order tuples in a relation to improve compression? B-Tree tuple grouping on a 2 integer attribute relation: Y Key = (X,Y) X

7 7 Database Talk COMPRESSING RELATIONS(File Level) R-Tree tuple grouping on a 2 integer attribute relation: Y X

8 COMPRESSING RELATIONS(File Level) Thus we can achieve efficient inter-page tuple placement by: ffl Using an existing sort order (single or multidimensional). ffl Applying B-Tree sort order when there is no pre-existing sort order. 8 Database Talk

9 COMPRESSING RELATIONS(Experiments) Our page level compression implementation had the following properties: ffl There was no slot array. ffl The page size was 4K. 9 Database Talk

10 10 Database Talk COMPRESSING RELATIONS(Experiments) Compression achieved on a catalog company's sales data: Size of Compressed Relation Over Size of Original Relation 100% 50% 40% 30% 20% 10% Size of Original Relation Random partitioning R-tree partitioning B-tree partitioning 0% Number of Attributes (Dimensionality) Figure 1: Varying tuple grouping

11 COMPRESSING RELATIONS(Experiments) achieved on synthetic data sets using R-Tree Compression ordering: Figure 2: Varying distribution: Uniform(left) and Exponential(right) 11 Database Talk Size of Compressed Relation Over Size of Original Relation 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% Original Relation Size Attributes range is 0-15 Attributes range is Attributes range is Attributes range is # Attributes (Dimensionality) Size of Compressed Relation Over Size of Original Relation 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% Original Relation Size Attributes range is 0-15 Attributes range is Attributes range is Attributes range is # Attributes (Dimensionality)

12 COMPRESSING RELATIONS(Experiments) Some interesting data points: ffl In memory throughput of compressed page interpretation was 15MB/s ( 40MB/s compressed) ffl A projection of several low cardinality fields from the sales data set compressed to 1/88th of its original size. 12 Database Talk

13 13 Database Talk COMPRESSING INDEXES(Pages) Must compress two types of index pages: ffl Leaf pages - Compression is identical to page level relational compression. ffl Internal pages - Can use lossy compression for greatly enhanced fanout. Y X

14 14 Database Talk COMPRESSING INDEXES(Pages) Approximating points using a coarse grid: estimating rectangle d Y 11 actual point 10 d Y actual point 01 d Y actual point estimated point d X d X d X Figure 3: Point approximation in lossy compression.

15 COMPRESSING INDEXES(Pages) Using lossy compression to estimate bounding boxes in internal nodes: ffl Store the bounding boxes conservatively in a coarse grid ffl When the frame of reference changes (on insert) reestimate the bounding boxes using the old estimates. ffl Whenever a child is followed, re-estimate the bounding box stored in the parent using FOR information in the child. 15 Database Talk

16 16 Database Talk COMPRESSING INDEXES(Experiments) Compressed R-Tree performance experiment: 90% I/O of query for compressed R-tree over I/O of query for uncompressed R-tree 80% 70% 60% 50% 40% 30% 20% 10% 0% point queries % 0.1% 1% 10% # attributes used for partial match query size of query region (percent of total size of region) Figure 4: Compressed R-trees on the Sales Dataset.

17 17 Database Talk COMPRESSING INDEXES(Experiments) Compressed R-Tree performace experiment: 90% I/O of query for compressed R-tree over I/O of query for uncompressed R-tree 80% 70% 60% 50% 40% 30% 20% point queries 1 partial match query 0.01% 0.1% 1% 10% size of query region (percent of total size of region) Compressed R-trees on the Tiger Orange County Figure 5: Dataset.

18 RELATED WORK ffl BTree prefix compression used in VSAM, Ingres etc... ffl Ng and Ravishankar ffl Sybase IQ gzip style compression 18 Database Talk

19 FUTURE WORK ffl Examine the application of FOR compression to interesting application areas (e.g. OLAP). ffl Extend the scheme to gracefully handle variable length data. 19 Database Talk

20 CONCLUSIONS ffl Developed a page at a time compression technique for fixed length tuple data which allows decompression on a per field per tuple granularity. ffl The technique can be applied to both the leaf and internal nodes of R-Trees and B-Trees. ffl Established a tie between indexing partitioning and inter-page compression optimization. ffl Presented experimental results indicating the current utility of these techniques. 20 Database Talk