CSE 530A. Query Planning. Washington University Fall 2013
|
|
- Donald Montgomery
- 5 years ago
- Views:
Transcription
1 CSE 530A Query Planning Washington University Fall 2013
2 Scanning When finding data in a relation, we've seen two types of scans Table scan Index scan There is a third common way Bitmap scan
3 Bitmap Scans When performing an index scan it is usually necessary to also retrieve the actual matching records from the table Most often, the records in the table file are not in the order accessed This leads to many random accesses in the table file Sequential file access is generally faster than random access If we could access the table file in sequentially then performance could be increased
4 Bitmap Scans In a bitmap scan, a bitmap is created while performing an index scan The bitmap indicates the position of the matching records in the table file (also called the heap) The index scan step is called a bitmap index scan A scan of the heap (table file) is then done, retrieving the records indicated by the bitmap The table scan step is called a bitmap heap scan
5 Lossy Bitmap Scans If a table is too large to efficiently create a bitmap of the records then a bitmap of the pages can be used The bitmap indicates which table pages contain records that match the predicate The bitmap heap scan then fetches the indicated pages in sequential order Must recheck the records in each fetched page to find the ones that match the predicate
6 Disjunctions Bitmap scans can be helpful in implementing queries with disjunctions If we have a query like " WHERE year < '2010' OR major = 'CSE'" and separate indexes on year and major then we could do an index scan on year to create the bitmap do an index scan on major and set the matching bits in the bitmap do the bitmap heap scan
7 Query Planning We now have three ways of scanning a relation table scan index scan bitmap index scan When should each method be used? Answer is not always obvious
8 Query Planning If we need to access every record in the relation Do a table scan. Accessing the index is just unnecessary overhead. If no appropriate index is available Table scan is only option. If index is available Best method depends on size of table and percentage of matching records If the table is small enough then it might be fastest to just do a table scan If the number of matching records is small then an index scan is probably fastest If the number of matching records is above some limit then a bitmap scan or table scan might be fastest
9 Query Planning What is the limit at which we should switch from index scan to bitmap or table scan? No hard and fast rule Some references say just 5% or greater matching records (Seems a little low to me)
10 Query Planning Three ways of scanning relations table scan index scan bitmap index scan Three ways of joining relations nested loop join merge join hash join
11 Query Planning Selecting from a single table gives 3 possible options 3 ways of scanning Joining two tables gives 3 3 options 3 for selecting from each table 3 for the actual join Each additional table joined multiplies the possibilities by for selecting from the additional table 3 for joining In addition, there are often multiple possibilities for when to handle the predicates How does the DBMS choose a query plan?
12 Query Planning A query plan can be represented as a tree leaves are scans inner nodes are joins, sorts, materializations nested loop join condition: students.mid = majors.mid table scan on students condition: year='2010' index scan on majors condition: title='cse'
13 Query Planning Some joins can be done in different orders For example, if we have x inner join y inner join z we'll get the same result if x join y is done first and then joined to z or y join z is done first and then joined to x join join join z x join x y y z
14 Query Planning It is the job of the query planner to pick the minimum cost plan Two issues The number of possible plans grows exponentially in the number of joined tables How is the cost of a plan calculated?
15 Query Planning We can assign a cost to a plan node if given The cost of the child nodes The size of the sets generated by the child nodes The cost of the actual operations of the node
16 Query Planning What do we mean by "cost"? Time! We're looking for the plan that takes the shortest time The cost of an operation is generally calculated as a function of the number of page fetches/writes and the CPU time spent I/O time usually dominates, but CPU time also plays a part Also often subdivided into sequential page and random page access as these can take different times Number of page accesses and CPU time directly dependent on size of data
17 Query Planning For a given plan node, the query planner tries to estimate both the cost (in some unit of time) and the size of the resulting set based on the type of operation and the size of the input sets The cost of a node in the plan tree includes the cost of its children So the cost of the root node is the total estimated cost of executing the plan
18 Query Planning The number of possible plans increases exponentially with the complexity of the query Quickly becomes impossible for a query planner to evaluate all possible plans The query planner itself needs to be fast Since the planner is choosing among various trees which often have identical subtrees, dynamic programming techniques (e.g., memoization) can be used Still may not have enough time to evaluate all possible plans
19 Query Planning Some query optimizers favor left-deep trees A left-deep tree adds each new table as the right child of a join left-deep right-deep bushy join join join join d a join join join join c b join a b c d a b c d
20 Query Planning The nodes can be made interchangeable by using an iterator interface Each child supplies the set resulting from its operation as an iterator to its parent Using an iterator interface also facilitates pipelining A parent node doesn't need to know if the child has a materialized set or is providing elements on demand
21 Query Planning Part of estimating the cost for a node requires estimating the number of records resulting from a scan or join For a select without a predicate we could use the number of rows in the table For an equals predicate on a unique column we know there will be at most one How do we know how many rows a scan with a predicate will return before actually doing the scan?
22 Query Planning The query planner tries to estimate the selection factor of predicates by using statistics on the distribution of values for each column Basically keeps a histogram for each column
23 Query Planning If the statistics kept by the DBMS for a column are significantly wrong then the performance of queries using that column can be greatly impacted The query planner could have vastly incorrect cost estimates and therefore pick a bad plan Tables with lots of modifications (inserts/updates/deletes) should periodically be analyzed The DBMS will scan the table and regenerate its histograms
24 Sargable Predicates A predicate is sargable if it does not interfere with the DBMS's ability to use an index Applying a function to a column in the WHERE clause usually makes it non-sargable Non-Sargable: WHERE year(date) = 2012 Sargable: WHERE date BETWEEN ' ' AND ' Non-Sargable: WHERE substring(model, 6) = 'Toyota' Sargable: WHERE model LIKE 'Toyota%' ("sargable" comes from Search ARGument ABLE)
25 Explain PostgreSQL has an EXPLAIN command that will give details on the chosen plan for a query Other DBMSs have similar features, but there is no standard EXPLAIN SELECT * FROM tenk1; QUERY PLAN Seq Scan on tenk1 (cost= rows=10000 width=244)
26 Explain EXPLAIN SELECT * FROM tenk1; QUERY PLAN Seq Scan on tenk1 (cost= rows=10000 width=244) The first cost value is the estimated start-up cost That is, the time before scan of the output can start The second cost value is the estimated cost to retrieve all the requested rows The time values are in arbitrary units based on the configured cost parameters (by default, in units of page fetches) The rows value is the estimated number of rows output by this node The width value is the estimated average width (in bytes) of the output rows
27 Explain EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 7000; QUERY PLAN Seq Scan on tenk1 (cost= rows=7033 width=244) Filter: (unique1 < 7000) Added a WHERE clause Number of rows has gone down Time has gone up slightly Reflects the extra CPU time used in checking the condition
28 Explain EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 100; QUERY PLAN Bitmap Heap Scan on tenk1 (cost= rows=106 width=244) Recheck Cond: (unique1 < 100) -> Bitmap Index Scan on tenk1_unique1 (cost= rows=106 width=0) Index Cond: (unique1 < 100) EXPLAIN outputs a textual representation of the plan tree The '->' indicates a child node and the level of indentation indicates the parent So in this example, the Bitmap Index Scan node is a child of the Bitmap Heap Scan node These generally always go together
29 Explain EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 100; QUERY PLAN Bitmap Heap Scan on tenk1 (cost= rows=106 width=244) Recheck Cond: (unique1 < 100) -> Bitmap Index Scan on tenk1_unique1 (cost= rows=106 width=0) Index Cond: (unique1 < 100) By changing the predicate, the estimated number of matching rows has dropped drastically which caused the planner to pick a different plan Note that the costs for the parent node includes the costs for the child nodes
30 Explain EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 3; QUERY PLAN Index Scan using tenk1_unique1 on tenk1 (cost= rows=2 width=244) Index Cond: (unique1 < 3) Changing the predicate again has again drastically reduced the estimated number of rows and changed the selected plan Why was table scan picked for the large number of rows? index scan picked for the small number of rows? bitmap scan picked for the medium number of rows?
31 Explain EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 3 AND stringu1 = 'xxx'; QUERY PLAN Index Scan using tenk1_unique1 on tenk1 (cost= rows=1 width=244) Index Cond: (unique1 < 3) Filter: (stringu1 = 'xxx'::name) Added a conjunction to the predicate Number of rows is smaller Cost is (very slightly) higher
32 Explain EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000; QUERY PLAN Bitmap Heap Scan on tenk1 (cost= rows=11 width=244) Recheck Cond: ((unique1 < 100) AND (unique2 > 9000)) -> BitmapAnd (cost= rows=11 width=0) -> Bitmap Index Scan on tenk1_unique1 (cost= rows=106 width=0) Index Cond: (unique1 < 100) -> Bitmap Index Scan on tenk1_unique2 (cost= rows=1042 width=0) Index Cond: (unique2 > 9000) In this example, the results of two bitmap index scans are being ANDed together before performing the bitmap heap scan Can be done since indexes exist on both columns Could also have been done by doing a bitmap scan on the one column and filtering by the other predicate
33 Explain EXPLAIN SELECT * FROM tenk1 t1, tenk2 t2 WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2; QUERY PLAN Nested Loop (cost= rows=106 width=488) -> Bitmap Heap Scan on tenk1 t1 (cost= rows=106 width=244) Recheck Cond: (unique1 < 100) -> Bitmap Index Scan on tenk1_unique1 (cost= rows=106 width=0) Index Cond: (unique1 < 100) -> Index Scan using tenk2_unique2 on tenk2 t2 (cost= rows=1 width=244) Index Cond: (t2.unique2 = t1.unique2) This example uses a nested loop join
34 Explain SET enable_nestloop = off; EXPLAIN SELECT * FROM tenk1 t1, tenk2 t2 WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2; QUERY PLAN Hash Join (cost= rows=106 width=488) Hash Cond: (t2.unique2 = t1.unique2) -> Seq Scan on tenk2 t2 (cost= rows=10000 width=244) -> Hash (cost= rows=106 width=244) -> Bitmap Heap Scan on tenk1 t1 (cost= rows=106 width=244) Recheck Cond: (unique1 < 100) -> Bitmap Index Scan on tenk1_unique1 (cost= rows=106 width=0) Index Cond: (unique1 < 100) Here's the same example with nested loop joins disabled Generally not a good idea to disable features like this. Note the increased query cost.
35 Explain Analyze EXPLAIN ANALYZE can be used to compare the estimated costs with the actual costs Note that EXPLAIN ANALYZE will cause the query to actually be executed Might want to perform in a transaction and rollback
36 Explain Analyze EXPLAIN ANALYZE SELECT * FROM tenk1 t1, tenk2 t2 WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2; QUERY PLAN Nested Loop (cost= rows=106 width=488) (actual time= rows=100 loops=1) -> Bitmap Heap Scan on tenk1 t1 (cost= rows=106 width=244) (actual time= rows=100 loops=1) Recheck Cond: (unique1 < 100) -> Bitmap Index Scan on tenk1_unique1 (cost= rows=106 width=0) (actual time= rows=100 loops=1) Index Cond: (unique1 < 100) -> Index Scan using tenk2_unique2 on tenk2 t2 (cost= rows=1 width=244) (actual time= rows=1 loops=100) Index Cond: (t2.unique2 = t1.unique2) Total runtime: ms The actual times are in milliseconds
CSE 530A. B+ Trees. Washington University Fall 2013
CSE 530A B+ Trees Washington University Fall 2013 B Trees A B tree is an ordered (non-binary) tree where the internal nodes can have a varying number of child nodes (within some range) B Trees When a key
More informationWhat s in a Plan? And how did it get there, anyway?
What s in a Plan? And how did it get there, anyway? Robert Haas 2018-06-01 2013 EDB All rights reserved. 1 Plan Contents: Structure Definition typedef struct Plan { NodeTag type; /* estimated execution
More informationReminders - IMPORTANT:
CMU - SCS 15-415/15-615 Database Applications Spring 2013, C. Faloutsos Homework 5: Query Optimization Released: Tuesday, 02/26/2013 Deadline: Tuesday, 03/19/2013 Reminders - IMPORTANT: Like all homework,
More informationNext-Generation Parallel Query
Next-Generation Parallel Query Robert Haas & Rafia Sabih 2013 EDB All rights reserved. 1 Overview v10 Improvements TPC-H Results TPC-H Analysis Thoughts for the Future 2017 EDB All rights reserved. 2 Parallel
More informationRelational Query Optimization. Overview of Query Evaluation. SQL Refresher. Yanlei Diao UMass Amherst October 23 & 25, 2007
Relational Query Optimization Yanlei Diao UMass Amherst October 23 & 25, 2007 Slide Content Courtesy of R. Ramakrishnan, J. Gehrke, and J. Hellerstein 1 Overview of Query Evaluation Query Evaluation Plan:
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY Database Systems: Fall 2009 Quiz I Solutions
Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.830 Database Systems: Fall 2009 Quiz I Solutions There are 15 questions and 12 pages in this quiz booklet.
More informationIntroduction to Database Systems CSE 344
Introduction to Database Systems CSE 344 Lecture 6: Basic Query Evaluation and Indexes 1 Announcements Webquiz 2 is due on Tuesday (01/21) Homework 2 is posted, due week from Monday (01/27) Today: query
More informationPostgreSQL Query Optimization. Step by step techniques. Ilya Kosmodemiansky
PostgreSQL Query Optimization Step by step techniques Ilya Kosmodemiansky (ik@) Agenda 2 1. What is a slow query? 2. How to chose queries to optimize? 3. What is a query plan? 4. Optimization tools 5.
More informationSQL QUERY EVALUATION. CS121: Relational Databases Fall 2017 Lecture 12
SQL QUERY EVALUATION CS121: Relational Databases Fall 2017 Lecture 12 Query Evaluation 2 Last time: Began looking at database implementation details How data is stored and accessed by the database Using
More informationReminders. Query Optimizer Overview. The Three Parts of an Optimizer. Dynamic Programming. Search Algorithm. CSE 444: Database Internals
Reminders CSE 444: Database Internals Lab 2 is due on Wednesday HW 5 is due on Friday Lecture 11 Query Optimization (part 2) CSE 444 - Winter 2017 1 CSE 444 - Winter 2017 2 Query Optimizer Overview Input:
More informationHash-Based Indexing 165
Hash-Based Indexing 165 h 1 h 0 h 1 h 0 Next = 0 000 00 64 32 8 16 000 00 64 32 8 16 A 001 01 9 25 41 73 001 01 9 25 41 73 B 010 10 10 18 34 66 010 10 10 18 34 66 C Next = 3 011 11 11 19 D 011 11 11 19
More informationAdvanced Database Systems
Lecture IV Query Processing Kyumars Sheykh Esmaili Basic Steps in Query Processing 2 Query Optimization Many equivalent execution plans Choosing the best one Based on Heuristics, Cost Will be discussed
More informationQuery Processing. Lecture #10. Andy Pavlo Computer Science Carnegie Mellon Univ. Database Systems / Fall 2018
Query Processing Lecture #10 Database Systems 15-445/15-645 Fall 2018 AP Andy Pavlo Computer Science Carnegie Mellon Univ. 2 ADMINISTRIVIA Project #2 Checkpoint #1 is due Monday October 9 th @ 11:59pm
More informationRelational Query Optimization. Overview of Query Evaluation. SQL Refresher. Yanlei Diao UMass Amherst March 8 and 13, 2007
Relational Query Optimization Yanlei Diao UMass Amherst March 8 and 13, 2007 Slide Content Courtesy of R. Ramakrishnan, J. Gehrke, and J. Hellerstein 1 Overview of Query Evaluation Query Evaluation Plan:
More informationPrinciples of Data Management. Lecture #12 (Query Optimization I)
Principles of Data Management Lecture #12 (Query Optimization I) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Notable News v B+ tree
More informationDATABASE PERFORMANCE AND INDEXES. CS121: Relational Databases Fall 2017 Lecture 11
DATABASE PERFORMANCE AND INDEXES CS121: Relational Databases Fall 2017 Lecture 11 Database Performance 2 Many situations where query performance needs to be improved e.g. as data size grows, query performance
More informationCSE 344 FEBRUARY 14 TH INDEXING
CSE 344 FEBRUARY 14 TH INDEXING EXAM Grades posted to Canvas Exams handed back in section tomorrow Regrades: Friday office hours EXAM Overall, you did well Average: 79 Remember: lowest between midterm/final
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY Database Systems: Fall 2017 Quiz I
Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.830 Database Systems: Fall 2017 Quiz I There are 15 questions and 12 pages in this quiz booklet. To receive
More informationOverview of Implementing Relational Operators and Query Evaluation
Overview of Implementing Relational Operators and Query Evaluation Chapter 12 Motivation: Evaluating Queries The same query can be evaluated in different ways. The evaluation strategy (plan) can make orders
More informationCSE 444: Database Internals. Lecture 11 Query Optimization (part 2)
CSE 444: Database Internals Lecture 11 Query Optimization (part 2) CSE 444 - Spring 2014 1 Reminders Homework 2 due tonight by 11pm in drop box Alternatively: turn it in in my office by 4pm, or in Priya
More informationHash table example. B+ Tree Index by Example Recall binary trees from CSE 143! Clustered vs Unclustered. Example
Student Introduction to Database Systems CSE 414 Hash table example Index Student_ID on Student.ID Data File Student 10 Tom Hanks 10 20 20 Amy Hanks ID fname lname 10 Tom Hanks 20 Amy Hanks Lecture 26:
More informationDatabase Systems CSE 414
Database Systems CSE 414 Lecture 10: Basics of Data Storage and Indexes 1 Reminder HW3 is due next Tuesday 2 Motivation My database application is too slow why? One of the queries is very slow why? To
More informationChapter 12: Query Processing. Chapter 12: Query Processing
Chapter 12: Query Processing Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 12: Query Processing Overview Measures of Query Cost Selection Operation Sorting Join
More informationChapter 12: Indexing and Hashing. Basic Concepts
Chapter 12: Indexing and Hashing! Basic Concepts! Ordered Indices! B+-Tree Index Files! B-Tree Index Files! Static Hashing! Dynamic Hashing! Comparison of Ordered Indexing and Hashing! Index Definition
More informationIntroduction to Database Systems CSE 414. Lecture 26: More Indexes and Operator Costs
Introduction to Database Systems CSE 414 Lecture 26: More Indexes and Operator Costs CSE 414 - Spring 2018 1 Student ID fname lname Hash table example 10 Tom Hanks Index Student_ID on Student.ID Data File
More informationParser: SQL parse tree
Jinze Liu Parser: SQL parse tree Good old lex & yacc Detect and reject syntax errors Validator: parse tree logical plan Detect and reject semantic errors Nonexistent tables/views/columns? Insufficient
More informationChapter 12: Indexing and Hashing
Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL
More informationChapter 12: Query Processing
Chapter 12: Query Processing Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Overview Chapter 12: Query Processing Measures of Query Cost Selection Operation Sorting Join
More informationQuery Processing & Optimization
Query Processing & Optimization 1 Roadmap of This Lecture Overview of query processing Measures of Query Cost Selection Operation Sorting Join Operation Other Operations Evaluation of Expressions Introduction
More informationAdministrivia. CS 133: Databases. Cost-based Query Sub-System. Goals for Today. Midterm on Thursday 10/18. Assignments
Administrivia Midterm on Thursday 10/18 CS 133: Databases Fall 2018 Lec 12 10/16 Prof. Beth Trushkowsky Assignments Lab 3 starts after fall break No problem set out this week Goals for Today Cost-based
More informationCS122 Lecture 15 Winter Term,
CS122 Lecture 15 Winter Term, 2014-2015 2 Index Op)miza)ons So far, only discussed implementing relational algebra operations to directly access heap Biles Indexes present an alternate access path for
More informationQuery Processing. Debapriyo Majumdar Indian Sta4s4cal Ins4tute Kolkata DBMS PGDBA 2016
Query Processing Debapriyo Majumdar Indian Sta4s4cal Ins4tute Kolkata DBMS PGDBA 2016 Slides re-used with some modification from www.db-book.com Reference: Database System Concepts, 6 th Ed. By Silberschatz,
More informationPS2 out today. Lab 2 out today. Lab 1 due today - how was it?
6.830 Lecture 7 9/25/2017 PS2 out today. Lab 2 out today. Lab 1 due today - how was it? Project Teams Due Wednesday Those of you who don't have groups -- send us email, or hand in a sheet with just your
More informationBecoming a better developer with explain
Becoming a better developer with explain Understanding Postgres Query planner Louise Grandjonc About me Louise Grandjonc (louise@ulule.com) Lead developer at Ulule (www.ulule.com) Python / Django developer
More informationIntroduction to Database Systems CSE 344
Introduction to Database Systems CSE 344 Lecture 10: Basics of Data Storage and Indexes 1 Student ID fname lname Data Storage 10 Tom Hanks DBMSs store data in files Most common organization is row-wise
More informationData Storage. Query Performance. Index. Data File Types. Introduction to Data Management CSE 414. Introduction to Database Systems CSE 414
Introduction to Data Management CSE 414 Unit 4: RDBMS Internals Logical and Physical Plans Query Execution Query Optimization Introduction to Database Systems CSE 414 Lecture 16: Basics of Data Storage
More informationCS222P Fall 2017, Final Exam
STUDENT NAME: STUDENT ID: CS222P Fall 2017, Final Exam Principles of Data Management Department of Computer Science, UC Irvine Prof. Chen Li (Max. Points: 100 + 15) Instructions: This exam has seven (7)
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Fall 2007 Lecture 9 - Query optimization
CSE 544 Principles of Database Management Systems Magdalena Balazinska Fall 2007 Lecture 9 - Query optimization References Access path selection in a relational database management system. Selinger. et.
More informationParallel Query In PostgreSQL
Parallel Query In PostgreSQL Amit Kapila 2016.12.01 2013 EDB All rights reserved. 1 Contents Parallel Query capabilities in 9.6 Tuning parameters Operations where parallel query is prohibited TPC-H results
More informationDesign and Implementation of Bit-Vector filtering for executing of multi-join qureies
Undergraduate Research Opportunity Program (UROP) Project Report Design and Implementation of Bit-Vector filtering for executing of multi-join qureies By Cheng Bin Department of Computer Science School
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2009 Lecture 6 - Storage and Indexing
CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2009 Lecture 6 - Storage and Indexing References Generalized Search Trees for Database Systems. J. M. Hellerstein, J. F. Naughton
More informationCMSC424: Database Design. Instructor: Amol Deshpande
CMSC424: Database Design Instructor: Amol Deshpande amol@cs.umd.edu Databases Data Models Conceptual representa1on of the data Data Retrieval How to ask ques1ons of the database How to answer those ques1ons
More informationReview. Relational Query Optimization. Query Optimization Overview (cont) Query Optimization Overview. Cost-based Query Sub-System
Review Relational Query Optimization R & G Chapter 12/15 Implementation of single Relational Operations Choices depend on indexes, memory, stats, Joins Blocked nested loops: simple, exploits extra memory
More informationRelational Query Optimization. Highlights of System R Optimizer
Relational Query Optimization Chapter 15 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Highlights of System R Optimizer v Impact: Most widely used currently; works well for < 10 joins.
More informationRelational Query Optimization
Relational Query Optimization Module 4, Lectures 3 and 4 Database Management Systems, R. Ramakrishnan 1 Overview of Query Optimization Plan: Tree of R.A. ops, with choice of alg for each op. Each operator
More informationAdvanced Databases: Parallel Databases A.Poulovassilis
1 Advanced Databases: Parallel Databases A.Poulovassilis 1 Parallel Database Architectures Parallel database systems use parallel processing techniques to achieve faster DBMS performance and handle larger
More informationCSE 444: Database Internals. Lectures 5-6 Indexing
CSE 444: Database Internals Lectures 5-6 Indexing 1 Announcements HW1 due tonight by 11pm Turn in an electronic copy (word/pdf) by 11pm, or Turn in a hard copy in my office by 4pm Lab1 is due Friday, 11pm
More informationR & G Chapter 13. Implementation of single Relational Operations Choices depend on indexes, memory, stats, Joins Blocked nested loops:
Relational Query Optimization R & G Chapter 13 Review Implementation of single Relational Operations Choices depend on indexes, memory, stats, Joins Blocked nested loops: simple, exploits extra memory
More informationQuerying Data with Transact SQL
Course 20761A: Querying Data with Transact SQL Course details Course Outline Module 1: Introduction to Microsoft SQL Server 2016 This module introduces SQL Server, the versions of SQL Server, including
More informationPrinciples of Data Management. Lecture #9 (Query Processing Overview)
Principles of Data Management Lecture #9 (Query Processing Overview) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Notable News v Midterm
More informationOn-Disk Bitmap Index Performance in Bizgres 0.9
On-Disk Bitmap Index Performance in Bizgres 0.9 A Greenplum Whitepaper April 2, 2006 Author: Ayush Parashar Performance Engineering Lab Table of Contents 1.0 Summary...1 2.0 Introduction...1 3.0 Performance
More informationDatabase System Concepts
Chapter 13: Query Processing s Departamento de Engenharia Informática Instituto Superior Técnico 1 st Semester 2008/2009 Slides (fortemente) baseados nos slides oficiais do livro c Silberschatz, Korth
More informationQuery Optimizer MySQL vs. PostgreSQL
Percona Live, Santa Clara (USA), 24 April 2018 Christian Antognini @ChrisAntognini antognini.ch/blog BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH
More informationCSE 544, Winter 2009, Final Examination 11 March 2009
CSE 544, Winter 2009, Final Examination 11 March 2009 Rules: Open books and open notes. No laptops or other mobile devices. Calculators allowed. Please write clearly. Relax! You are here to learn. Question
More informationQuery Processing: A Systems View. Announcements (March 1) Physical (execution) plan. CPS 216 Advanced Database Systems
Query Processing: A Systems View CPS 216 Advanced Database Systems Announcements (March 1) 2 Reading assignment due Wednesday Buffer management Homework #2 due this Thursday Course project proposal due
More informationChapter 13: Query Processing
Chapter 13: Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 13.1 Basic Steps in Query Processing 1. Parsing
More informationCSE 530A. Inheritance and Partitioning. Washington University Fall 2013
CSE 530A Inheritance and Partitioning Washington University Fall 2013 Inheritance PostgreSQL provides table inheritance SQL defines type inheritance, PostgreSQL's table inheritance is different A table
More informationSorting and Searching
Sorting and Searching Lecture 2: Priority Queues, Heaps, and Heapsort Lecture 2: Priority Queues, Heaps, and Heapsort Sorting and Searching 1 / 24 Priority Queue: Motivating Example 3 jobs have been submitted
More informationQUERY OPTIMIZATION E Jayant Haritsa Computer Science and Automation Indian Institute of Science. JAN 2014 Slide 1 QUERY OPTIMIZATION
E0 261 Jayant Haritsa Computer Science and Automation Indian Institute of Science JAN 2014 Slide 1 Database Engines Main Components Query Processing Transaction Processing Access Methods JAN 2014 Slide
More informationIntro to DB CHAPTER 12 INDEXING & HASHING
Intro to DB CHAPTER 12 INDEXING & HASHING Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing
More informationChapter 11: Indexing and Hashing
Chapter 11: Indexing and Hashing Basic Concepts Ordered Indices B + -Tree Index Files B-Tree Index Files Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing Index Definition in SQL
More informationQuery Processing: The Basics. External Sorting
Query Processing: The Basics Chapter 10 1 External Sorting Sorting is used in implementing many relational operations Problem: Relations are typically large, do not fit in main memory So cannot use traditional
More informationFundamentals of Database Systems
Fundamentals of Database Systems Assignment: 4 September 21, 2015 Instructions 1. This question paper contains 10 questions in 5 pages. Q1: Calculate branching factor in case for B- tree index structure,
More informationQuery optimization. Elena Baralis, Silvia Chiusano Politecnico di Torino. DBMS Architecture D B M G. Database Management Systems. Pag.
Database Management Systems DBMS Architecture SQL INSTRUCTION OPTIMIZER MANAGEMENT OF ACCESS METHODS CONCURRENCY CONTROL BUFFER MANAGER RELIABILITY MANAGEMENT Index Files Data Files System Catalog DATABASE
More informationProblem Set 2 Solutions
6.893 Problem Set 2 Solutons 1 Problem Set 2 Solutions The problem set was worth a total of 25 points. Points are shown in parentheses before the problem. Part 1 - Warmup (5 points total) 1. We studied
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY Database Systems: Fall 2015 Quiz I
Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.830 Database Systems: Fall 2015 Quiz I There are 12 questions and 13 pages in this quiz booklet. To receive
More informationQuery Processing: an Overview. Query Processing in a Nutshell. .. CSC 468 DBMS Implementation Alexander Dekhtyar.. QUERY. Parser.
.. CSC 468 DBMS Implementation Alexander Dekhtyar.. Query Processing: an Overview Query Processing in a Nutshell QUERY Parser Preprocessor Logical Query plan generator Logical query plan Query rewriter
More informationSorting and Searching
Sorting and Searching Lecture 2: Priority Queues, Heaps, and Heapsort Lecture 2: Priority Queues, Heaps, and Heapsort Sorting and Searching 1 / 24 Priority Queue: Motivating Example 3 jobs have been submitted
More information! A relational algebra expression may have many equivalent. ! Cost is generally measured as total elapsed time for
Chapter 13: Query Processing Basic Steps in Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 1. Parsing and
More informationChapter 13: Query Processing Basic Steps in Query Processing
Chapter 13: Query Processing Basic Steps in Query Processing! Overview! Measures of Query Cost! Selection Operation! Sorting! Join Operation! Other Operations! Evaluation of Expressions 1. Parsing and
More informationCS330. Query Processing
CS330 Query Processing 1 Overview of Query Evaluation Plan: Tree of R.A. ops, with choice of alg for each op. Each operator typically implemented using a `pull interface: when an operator is `pulled for
More information2. Make an input file for Query Execution Steps for each Q1 and RQ respectively-- one step per line for simplicity.
General Suggestion/Guide on Program (This is only for suggestion. You can change your own design as needed and you can assume your own for simplicity as long as it is reasonable to make it as assumption.)
More informationColumn Stores vs. Row Stores How Different Are They Really?
Column Stores vs. Row Stores How Different Are They Really? Daniel J. Abadi (Yale) Samuel R. Madden (MIT) Nabil Hachem (AvantGarde) Presented By : Kanika Nagpal OUTLINE Introduction Motivation Background
More informationQuery Optimizer MySQL vs. PostgreSQL
Percona Live, Frankfurt (DE), 7 November 2018 Christian Antognini @ChrisAntognini antognini.ch/blog BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART
More informationSchema for Examples. Query Optimization. Alternative Plans 1 (No Indexes) Motivating Example. Alternative Plans 2 With Indexes
Schema for Examples Query Optimization (sid: integer, : string, rating: integer, age: real) (sid: integer, bid: integer, day: dates, rname: string) Similar to old schema; rname added for variations. :
More informationCopyright 2016 Ramez Elmasri and Shamkant B. Navathe
CHAPTER 19 Query Optimization Introduction Query optimization Conducted by a query optimizer in a DBMS Goal: select best available strategy for executing query Based on information available Most RDBMSs
More informationOverview of Query Evaluation. Overview of Query Evaluation
Overview of Query Evaluation Chapter 12 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Overview of Query Evaluation v Plan: Tree of R.A. ops, with choice of alg for each op. Each operator
More informationRelational Query Optimization
Relational Query Optimization Chapter 15 Ramakrishnan & Gehrke (Sections 15.1-15.6) CPSC404, Laks V.S. Lakshmanan 1 What you will learn from this lecture Cost-based query optimization (System R) Plan space
More informationDatabase Systems CSE 414
Database Systems CSE 414 Lecture 15-16: Basics of Data Storage and Indexes (Ch. 8.3-4, 14.1-1.7, & skim 14.2-3) 1 Announcements Midterm on Monday, November 6th, in class Allow 1 page of notes (both sides,
More informationAnnouncement. Reading Material. Overview of Query Evaluation. Overview of Query Evaluation. Overview of Query Evaluation 9/26/17
Announcement CompSci 516 Database Systems Lecture 10 Query Evaluation and Join Algorithms Project proposal pdf due on sakai by 5 pm, tomorrow, Thursday 09/27 One per group by any member Instructor: Sudeepa
More informationD B M G Data Base and Data Mining Group of Politecnico di Torino
Database Management Data Base and Data Mining Group of tania.cerquitelli@polito.it A.A. 2014-2015 Optimizer operations Operation Evaluation of expressions and conditions Statement transformation Description
More informationCS122 Lecture 4 Winter Term,
CS122 Lecture 4 Winter Term, 2014-2015 2 SQL Query Transla.on Last time, introduced query evaluation pipeline SQL query SQL parser abstract syntax tree SQL translator relational algebra plan query plan
More informationQuery Optimization. Schema for Examples. Motivating Example. Similar to old schema; rname added for variations. Reserves: Sailors:
Query Optimization Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Schema for Examples (sid: integer, sname: string, rating: integer, age: real) (sid: integer, bid: integer, day: dates,
More informationRELATIONAL OPERATORS #1
RELATIONAL OPERATORS #1 CS 564- Spring 2018 ACKs: Jeff Naughton, Jignesh Patel, AnHai Doan WHAT IS THIS LECTURE ABOUT? Algorithms for relational operators: select project 2 ARCHITECTURE OF A DBMS query
More informationChapter 12: Indexing and Hashing
Chapter 12: Indexing and Hashing Database System Concepts, 5th Ed. See www.db-book.com for conditions on re-use Chapter 12: Indexing and Hashing Basic Concepts Ordered Indices B + -Tree Index Files B-Tree
More informationOverview of Query Evaluation
Overview of Query Evaluation Chapter 12 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Overview of Query Evaluation Plan: Tree of R.A. operators, with choice of algorithm for each operator.
More informationPostgreSQL: Decoding Partition
PostgreSQL: Decoding Partition Beena Emerson February 14, 2019 1 INTRODUCTION 2 What is Partitioning? Why partition? When to Partition? What is Partitioning? Subdividing a database table into smaller parts.
More informationIntroduction to Query Processing and Query Optimization Techniques. Copyright 2011 Ramez Elmasri and Shamkant Navathe
Introduction to Query Processing and Query Optimization Techniques Outline Translating SQL Queries into Relational Algebra Algorithms for External Sorting Algorithms for SELECT and JOIN Operations Algorithms
More information6.830 Problem Set 2 (2017)
6.830 Problem Set 2 1 Assigned: Monday, Sep 25, 2017 6.830 Problem Set 2 (2017) Due: Monday, Oct 16, 2017, 11:59 PM Submit to Gradescope: https://gradescope.com/courses/10498 The purpose of this problem
More informationDatabase Systems CSE 414
Database Systems CSE 414 Lecture 10-11: Basics of Data Storage and Indexes (Ch. 8.3-4, 14.1-1.7, & skim 14.2-3) 1 Announcements No WQ this week WQ4 is due next Thursday HW3 is due next Tuesday should be
More informationLeveraging Query Parallelism In PostgreSQL
Leveraging Query Parallelism In PostgreSQL Get the most from intra-query parallelism! Dilip Kumar (Principle Software Engineer) Rafia Sabih (Software Engineer) 2013 EDB All rights reserved. 1 Overview
More informationDatabase Tuning and Physical Design: Basics of Query Execution
Database Tuning and Physical Design: Basics of Query Execution Spring 2018 School of Computer Science University of Waterloo Databases CS348 (University of Waterloo) Query Execution 1 / 43 The Client/Server
More information7. Query Processing and Optimization
7. Query Processing and Optimization Processing a Query 103 Indexing for Performance Simple (individual) index B + -tree index Matching index scan vs nonmatching index scan Unique index one entry and one
More informationPhysical DB design and tuning: outline
Physical DB design and tuning: outline Designing the Physical Database Schema Tables, indexes, logical schema Database Tuning Index Tuning Query Tuning Transaction Tuning Logical Schema Tuning DBMS Tuning
More informationIndexing. Week 14, Spring Edited by M. Naci Akkøk, , Contains slides from 8-9. April 2002 by Hector Garcia-Molina, Vera Goebel
Indexing Week 14, Spring 2005 Edited by M. Naci Akkøk, 5.3.2004, 3.3.2005 Contains slides from 8-9. April 2002 by Hector Garcia-Molina, Vera Goebel Overview Conventional indexes B-trees Hashing schemes
More informationInformatik Vertiefung Database Systems
Noah Berni Informatik Vertiefung Database Systems April 2017 University of Zurich Department of Informatics (IFI) Binzmühlestrasse 14, CH-8050 Zürich, Switzerland 2 3 1 Introduction The main target of
More informationQuery Parallelism In PostgreSQL
Query Parallelism In PostgreSQL Expectations and Opportunities 2013 EDB All rights reserved. 1 Dilip Kumar (Principle Software Engineer) Rafia Sabih (Software Engineer) Overview Infrastructure for parallelism
More information6.830 Lecture 8 10/2/2017. Lab 2 -- Due Weds. Project meeting sign ups. Recap column stores & paper. Join Processing:
Lab 2 -- Due Weds. Project meeting sign ups 6.830 Lecture 8 10/2/2017 Recap column stores & paper Join Processing: S = {S} tuples, S pages R = {R} tuples, R pages R < S M pages of memory Types of joins
More informationJoin (SQL) - Wikipedia, the free encyclopedia
페이지 1 / 7 Sample tables All subsequent explanations on join types in this article make use of the following two tables. The rows in these tables serve to illustrate the effect of different types of joins
More informationPhysical Design. Elena Baralis, Silvia Chiusano Politecnico di Torino. Phases of database design D B M G. Database Management Systems. Pag.
Physical Design D B M G 1 Phases of database design Application requirements Conceptual design Conceptual schema Logical design ER or UML Relational tables Logical schema Physical design Physical schema
More information