An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

Transcription

1 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database Stephan Müller, Hasso Plattner Enterprise Platform and Integration Concepts Hasso Plattner Institute, Potsdam (Germany) DOLAP 0 November nd

2 Agenda Data aggregation Aggregations in Hyrise Cost factors and benchmarks Summary and future work DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

3 Motivation Aggregation Abstractions for conceptualizing the real world Resource-intensive operation in OLAP workloads Columnar in-memory databases Optimized for set processing Enables reunification of OLTP and OLAP [] [] H. Plattner. A common database approach for OLTP and OLAP using an in-memory column database. In SIGMOD, 009. DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

4 Data Aggregation SELECT product, SUM (amount) AS revenue FROM sales WHERE year=0 GROUP BY product Two phases Grouping (hash-based, sorting, nested loops) Calculation DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

5 Data Aggregation in Hyrise Columnar Storage Dictionary compression id fruit amount id fruit amount 6 7 apples bananas cherries apples cherries melons bananas "fruit" dictionary apples bananas cherries grapes melons 8 grapes 0 8 DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

6 Data Aggregation Grouping table id fruit amount position on "fruit" 0,, 6, dictionary apples bananas cherries grapes melons 8 DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

7 Data Aggregation - Calculation id table fruit amount "amount" dictionary position on "fruit" 0,, 6, fruit apples bananas cherries grapes melons sum(amount) DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

8 Cost Modeling in IMDBs Cost model based on cache misses [] Basic patterns for each cache level Combine patterns to model complex operations [] Manegold, S. et al. 00. Generic database cost models for hierarchical memory systems. VLDB. (00). DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

9 Cost Modeling of the Aggregation DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

10 Cost Factors Data-dependent Dataset size Distinct grouping values Distinct grouping values distribution Sorting of grouping values Data-independent Aggregation function Hash implementation DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

11 Hash Implementations Ordering Sorted tree (std::map) Unsorted hash table (std::unordered_map) Pre-allocation strategies Naïve implementation (automatic growth) Distinct value setup (scaffold of hash map) Histogram-based setup (scaffold and position list) DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

12 Data Set Size Number of bytes processed during query Input table data Intermediate result data Final result data Influencing factors Relation size Relational expression Grouping attributes Aggregation attributes Aggregation functions DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

13 Distinct Values L Cache Misses DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

14 Distinct Values CPU Cycles DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

15 Distinct Values Distribution Number of cache misses 0e+00 e+06 e+06 6e+06 8e+06 Uniform L cache misses Exponential L cache misses Uniform L cache misses Exponential L cache misses Number of distinct values DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

16 Impact of Sorted Attributes L Misses DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

17 Hash Implementations L Misses DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

18 Hash Implementations CPU Cycles DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

19 Summary High impact of aggregation operation in a mixed OLTP and OLAP workload Data characteristics influence aggregation performance Hash implementation choice matters Pre-allocation can reduce cache misses Future work Multithreaded aggregations Extension of existing cost models DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database

20 0 Thank you! DOLAP 0 An In-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database