COLUMN DATABASES A NDREW C ROTTY & ALEX G ALAKATOS

Transcription

1 COLUMN DATABASES A NDREW C ROTTY & ALEX G ALAKATOS

2 OUTLINE RDBMS SQL Row Store Column Store C-Store Vertica MonetDB Hardware Optimizations

3 FACULTY MEMBER VERSION

4 EXPERIMENT Question: How does time spent as a faculty member affect one's attention span?

5 HYPOTHESIS Attention Span vs Time as Faculty Attention Span 0 < 1 10 > 100 Time as Faculty (years)

6 WHAT IS AN RDBMS? Relational database management system Standard for information storage/retrieval Based on Codd's relational model Data structured as relations Manipulated using relational operators Structure specified by a schema

7 EXAMPLE id name dept salary 1756 Scott Physics 50k 1757 Bob Math 60k 1758 John CS 80k

8 WHAT IS SQL? Structured Query Language Used to operate on relations Four main operations: select insert update delete

9 SELECT STATEMENT select <column(s)> from <table> where <column> = <val>

10 EXAMPLE id name dept salary 1756 Scott Physics 50k 1757 Bob Math 60k 1758 John CS 80k

11 ROW STORE id name dept salary id name dept salary 1756 Scott Physics 50k 1757 Bob Math 60k 1758 John CS 80k 1756 Scott Physics 50k id name dept salary 1757 Bob Math 60k id name dept salary 1758 John CS 80k

12 ROW STORE What is the name, department, and salary of the employee with id 1757? select name, dept, salary from employee where id = 1757

13 ROW STORE id name dept salary 1756 Scott Physics 50k select name, dept, salary from employee where id = 1757 id name dept salary 1757 Bob Math 60k id name dept salary 1758 John CS 80k

14 ROW STORE What is the average salary of all employees? select avg(salary) from employee

15 ROW STORE id name dept salary 1756 Scott Physics 50k select avg(salary) from employee id name dept salary 1757 Bob Math 60k id name dept salary 1758 John CS 80k

16 ROW STORE id name dept salary 1756 Scott Physics 50k select avg(salary) from employee id name dept salary 1757 Bob Math 60k id name dept salary 1758 John CS 80k

17 COLUMN STORE id name dept salary 1756 Scott Physics 50k 1757 Bob Math 60k 1758 John CS 80k id dept Physics Math CS name Scott Bob John salary 50k 60k 80k

18 COLUMN STORE What is the average salary of all employees? select avg(salary) from employee

19 COLUMN STORE select avg(salary) from employee id dept Physics Math CS name Scott Bob John salary 50k 60k 80k

20 HISTORY TAXIR 1969 Biology information retrieval RAPID 1976 Statistics Canada Canadian Census of Population and Housing Sybase IQ Early 1990s Sybase (later SAP) Only commercial column DB for many years

21 C-STORE Project from Brown, Brandeis, MIT, and UMass Boston Read-optimized Contributions: Hybrid architecture Novel data structures Advanced compression 2005: Winner "Best Logo" Award

22 ARCHITECTURE 2005: Winner "Most Intricate Architecture" Award

23 What does it do??? TUPLE MOVER

24 TUPLE MOVER What does it do??? Moves tuples 2005: Winner "Most Creative Name" Award

25 PROJECTIONS id name dept salary 1756 Scott Physics 50k 1757 Bob Math 60k 1758 John CS 80k name John Bob Scott name Scott Bob John dept CS Math Physics salary 50k 60k 80k

26 JOIN INDEXES name dept John CS Bob Math id name dept salary 1756 Scott Physics 50k 1757 Bob Math 60k 1758 John CS 80k Scott name Physics salary id Scott 50k Bob 60k John 80k

27 COMPRESSION Similar data types Less information = less I/O Operate on compressed data Compression strategies: Run length encoding (v,f,n) Delta encoding Bitmap encoding (v,b) Distinct value encoding

28 RUN LENGTH ENCODING Original gender female male male male male female female female Result gender (f, 0, 1) (m, 1, 4) (f, 5, 3)

29 DELTA ENCODING Original salary 70,000 70,500 70,900 71,250 75,000 79,000 81,500 82,000 Result salary 70, ,250 3,500 2,

30 BITMAP ENCODING Original dept CS Math Math English CS CS Biology Math Result dept bitmap CS Math English Biology

31 DISTINCT VALUE ENCODING Original gender age female 5 male 4 male 4 male 8 female 4 female 5 Stage 1 gender age Stage 2 gender/age

32 LATE MATERIALIZATION Tuple reconstruction is: Costly Often unnecessary Solution: Apply predicates early per column Reconstruct tuples as late as possible Bring in only relevant columns Also improves cache performance

33 VERTICA Commercialization of C-Store BI and analytics market Acquired by HP in 2011 Differences from C-Store Automatic layout tool No join indexes Instead requires super projection

34 MONETDB Developed at CWI (Netherlands) High performance on complex queries Column store architecture Uses demand paging Exploits CPU caches Automatic/self-tuning indexes Open source!

35 HARDWARE OPTIMIZATIONS Ocelot Hardware-oblivious abstraction Built on top of MonetDB Easily and efficiently support operations on: CPU GPU FPGA Take advantage of optimal memory access patterns CPU: prefetching, cache awareness GPU: coalesce memory accesses

36 HARDWARE OPTIMIZATIONS

37 SUMMARY RDBMS SQL Row Store Column Store C-Store Vertica MonetDB Hardware Optimizations Questions?