Access Path Selection in Main-Memory Optimized Data Systems

Transcription

1 Access Path Selection in Main-Memory Optimized Data Systems Should I Scan or Should I Probe? Manos Athanassoulis Harvard University Talk at CS265, February 16 th,

2 Access Path Selection SELECT x FROM table_a WHERE y < 10;...the system chooses how to retrieve it. 3

3 Access Path Choices Full Base Data Scan Secondary Index Scan (Auxilliary copy of the data + structure) 4

4 Access Path Choices Full Base Data Scan Secondary Index Scan (Auxilliary copy of the data + structure) 5

5 Selectivity Scan is best 0% Index is best why ask this question anew? 6

6 new workloads new architectures new hardware more and more concurrent similar read queries columnar data organization main-memory optimized data systems 7

7 modern data systems for analytics 8

8 Column(-group) Orientation Compression 9

9 SIMD Multi-Core Vectorized Processing C 1 C 2 C 3 C 4 10

10 1:1 Scan to Result Shared Scans S 1 S 2 S 3 S 1 r 1 r 2 r 3 r 1 r 2 r 3 11

11 Q 0, Q 1, Q 2,...Q n 12

12 ? Are indexes ever needed? Q 0, Q 1, Q 2,...Q n 13

13 ? Are indexes ever needed? 14

14 If so, how should the optimizer choose an access path? Q 0, Q 1, Q 2,...Q n 15

15 !"# ratio = +,-./ 0123 #45, 0123 > 1 scan < 1 index let us model 17

16 access path selection modeling ( &* +, *.0 rows tuple size memory bandwidth query selectivity scan data -.*/0&,12&. = *.0 ) ( ) &* +, scan data!"#$%#&# = ( ) &* +, what if we have q queries? 18

17 access path selection modeling q concurrent queries * &, -., rows tuple size memory bandwidth query selectivity queries predicate evaluation cost (CPU) scan data!"#$%#&# = (#) * + &, -., * CPU cost per query (may be too high!) scan data 23,45&.67&3 = 9 ; :<= selectivity per query,35 : + * + &, =! -. ; + * + &, ; -.,,. &.! ; = 9,35 : :<= total query selectivity (sum) 19

18 !"#$%&'( = *#+, - ('./,, ! 4 -, - ('./ + +!"#$;#(# <96=:"#(6>?#8 56'78(/9:(6 20

19 access path selection modeling index data traverse tree traverse leaves+data result write./&&./01&/#0' = '34 5! 6 +,! "# $% #&' ( ) * +, -"# 7&01&#./01&/#0' = #&' 6! * 6 +, 80"0./01&/#0' = #&' 6! 6 -"# $% rows tuple size memory bandwidth query selectivity queries predicate evaluation cost (CPU) branching factor (fanout) cache miss latency index tuple size how to make index drop-in replacement? 9&#:'"%/-"& = #&' 6! 6 "# $% 21

20 index data traverse tree traverse leaves+data ;3/" = #&' 6! 6 '34 < #&' 6! 6 -"# $% access path selection modeling./&&./01&/#0' = '34 5! 6 +,! "# $% #&' ( ) * +, -"# 7&01&#./01&/#0' = #&' 6! * 6 +, 80"0./01&/#0' = #&' 6! 6 -"# $% sort result write 9&#:'"%/-"& = #&' 6! 6 "# $% rows tuple size memory bandwidth query selectivity queries predicate evaluation cost (CPU) branching factor (fanout) cache miss latency index tuple size how to make index drop-in replacement? 22

21 !"#$%&'() = +, -'. / 0, & , 0 5, & 1 + 0, 6)( , )( :$$9:;<$:(;- =$;<$(9:;<$:(;- >;);9:;<$:(;-?$(@-)8:6)$ 23

22 new access path selection index datares res res index design data size & hardware resdatares res!"# ratio = +, -.//-.01/ # 5, 6/01/2-.01/ / /2:38;.<8/ =0> #?0@7080, B"C?D28 DE + F./G<?08/2 + # 5, 9/2:38;.<8/ scan Dynamic Parameter + #concurrent read queries # 5 5 sum of query selectivity of q queries # 5 = 2/3 I IJK data size (for result) 29

23 new access path selection index datares res res!"# ratio = +, -./ 1 0 1, # 5, 6, , 789 :; + 1, 89 :; <=> 1, 89 :;, 1 + # 5, 1, 89 :; 1 89 :; 9B rows tuple size memory bandwidth query selectivity queries predicate evaluation cost (CPU) branching factor (fanout) cache miss latency index tuple size scan resdatares res Dynamic Parameter + #concurrent read queries # 5 5 sum of query selectivity of q queries # 5 = 9B- C CDE 30

24 new access path selection index datares res res!"# ratio = data size +, -./ index design, 23, # 7, hardware characteristics data layout 23, <=> 9:, 23 + # 7, 9: + 9: + ;9: 1 9: 23 :B ;9: rows tuple size memory bandwidth query selectivity queries predicate evaluation cost (CPU) branching factor (fanout) cache miss latency index tuple size scan resdatares res Dynamic Parameter + #concurrent read queries # 7 7 sum of query selectivity of q queries # 7 = :B- C CDE 31

25 new access path selection Modeling Index 3 Scan 2 Scan Index data size has complex effect concurrency matters!! 32

26 new access path selection Experiments 8 Queries 8 Cores 100M Tuples 8 Cores Selectivity Crossover Scan Index Selectivity Crossover Index Scan shared scans help up to a point Relation Size Concurrent read queries 33

27 new access path selection Experiments 8 Queries 8 Cores 100M Tuples 8 Cores Selectivity Crossover Scan Index Selectivity Crossover Scan shared scans help up to a point Index Relation Size Concurrent read queries 34

28 new access path selection Experiments 8 Queries 8 Cores 100M Tuples 8 Cores Selectivity Crossover Scan Index Selectivity Crossover Index Scan Two x 256 Relation Size Concurrent read queries 35

29 new access path selection Selectivity Scan is best Selectivity Scan is best 0% Index is best 0% Index is best Concurrent Queries 36

30 10% Hardware Improvements 1% 0% Dawn of time

31 10% Hardware Improvements Column Stores 1% 0% Dawn of time

32 10% Hardware Improvements Column Stores 1% Main Memory 0% Dawn of time

33 10% Hardware Improvements Column Stores What-if 1% questions Main Memory 0% Dawn of time Future 40