class 13 scans vs indexes prof. Stratos Idreos

Size: px

Start display at page:

Download "class 13 scans vs indexes prof. Stratos Idreos"

Anabel Cameron
5 years ago
Views:

1 class 13 scans vs indexes prof. Stratos Idreos

2 b-tree - dynamic tree - always balanced 35,50 35, 12,20 50, 1,2,3 12,15,17 20, Stratos Idreos 2 /24

3 select from R where A<v and. (secondary) index vs scan: the eternal battle A A Stratos Idreos 3 /24

4 select from R where A<v and. (secondary) index vs scan: the eternal battle A A Just having indexes in the system is or can be useless or even bad for performance Knowing when to use an index is key Stratos Idreos 3 /24

5 select from R where A<v and. Primary index vs secondary vs scan? (secondary) index vs scan: the eternal battle A A Just having indexes in the system is or can be useless or even bad for performance Knowing when to use an index is key Stratos Idreos 3 /24

6 applications sql algorithms/operators database kernel design/implement numerous possible algorithms + data representations choose the best data source, algorithms and path for each query data data data Stratos Idreos 4 /24

7 scan secondary index scan Stratos Idreos 5 /24

8 scan secondary index scan Stratos Idreos 5 /24

9 A B C a1 a2 a3 a4 a5 b1 b2 b3 b4 b5 c1 c2 c3 c4 c5 A a5 a3 a2 a1 a4 secondary index on A values out of order with base data a query that select on A and then needs B intermediate out of order Stratos Idreos 6 /24

10 A B C A a1 a2 a3 a4 a5 b1 b2 b3 b4 b5 c1 c2 c3 c4 c a5 a3 a2 a1 a4 secondary index on A values out of order with base data a query that select on A and then needs B intermediate out of order Stratos Idreos 6 /24

11 A B C A a1 a2 a3 a4 a5 b1 b2 b3 b4 b5 c1 c2 c3 c4 c a5 a3 a2 a1 a4 secondary index on A values out of order with base data a query that select on A and then needs B A a5 a3 a2 a1 a4 select Stratos Idreos B b1 b2 b3 b4 b5 intermediate out of order /24

12 covering index: contains all columns needed for a set of queries A A B no need to go to base data but Stratos Idreos 7 /24

13 A A Vs. random access to traverse the tree & need to sort result sequential access pattern but needs to access all data Stratos Idreos 8 /24

14 Stratos Idreos 9 /24

15 the standard solution 1) maintain statistics, 2) optimizer chooses access path depending on estimated selectivity Stratos Idreos 10/24

16 what is wrong with that the standard solution 1) maintain statistics, 2) optimizer chooses access path depending on estimated selectivity Stratos Idreos 10/24

17 Execution time (sec) Index Scan Full Scan Result selectivity (%) Normalized execution time Original Tuned Q1 Q3 Q5 Q7 Q9 Q11 Q13 Q16 Q19 Q22 TPC-H Query Stratos Idreos 11/24

18 Execution time (sec) Index Scan Full Scan Result selectivity (%) Normalized execution time Original Tuned Q1 Q3 Q5 Q7 Q9 Q11 Q13 Q16 Q19 Q22 TPC-H Query ROBUSTNESS Stratos Idreos 11/24

19 can we just recompute the statistics? Execution time (sec) basic stats per column for pair Full Scan 200 Index Scan Optimizer decision Avg. statistics collection Result selectivity (%) Result selectivity (%) Result selectivity (%) Stratos Idreos 12/24

20 2012, somewhere in Germany if I keep 30 data systems researchers trapped in a castle for a week, we might be able to define robust query processing and find a few solutions Stratos Idreos 13/24

21 robust query processing (best definition to date by Goetz) graceful degradation when the environment changes 14 Stratos Idreos 14/24

22 Renata Borovica University of Melbourne Marcin Zukowski Snowflake Campbell Fraser Google Can we avoid bad access path selection (secondary index vs scan) when we have stale (or no) statistics? index response time scan Stratos Idreos selectivity 15/24

23 select min(a) from R where B<10 and C<80 logical plan optimizer mid query reoptimization physical plan execution Stratos Idreos 16/24

24 Index Scan Cost ~ TS cost TS cost Switch Scan Smooth Scan Table Scan SWITCH SCAN while index probing switch to scan if cardinality > estimation good: avoids worst case bad: performance cliff SMOOTH SCAN goal avoid performance cliff close to optimal X EC Cardinality Design smooth scan Stratos Idreos 17/24

25 Stratos Idreos 18/24

26 Stratos Idreos 18/24

27 Read: Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe? Mike. Kester, Manos Athanassoulis, and Stratos Idreos ACM SIGMOD International Conference on Management of Data, 2017 Browse: Smooth Scan: Statistics-Oblivious Access Paths Renata Borovica, Stratos Idreos, Anastasia Ailamaki, Marcin Zukowski and Campbell Fraser IEEE International Conference on Data Engineering (ICDE), 2015 Extra: Efficient mid-query re-optimization of sub-optimal query execution plans Navin Kabra and David DeWitt ACM SIGMOD International Conference on Management of Data, 1998 Stratos Idreos 19/24

28 class 13 scans vs indexes DATA SYSTEMS prof. Stratos Idreos

Smooth Scan: Statistics-Oblivious Access Paths. Renata Borovica-Gajic Stratos Idreos Anastasia Ailamaki Marcin Zukowski Campbell Fraser

Smooth Scan: Statistics-Oblivious Access Paths Renata Borovica-Gajic Stratos Idreos Anastasia Ailamaki Marcin Zukowski Campbell Fraser Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q16 Q18 Q19 Q21 Q22