1 Algebraic Query Optimization

Transcription

1 1 Algebraic Query Optimization 1.1 Relational Query Languages We have encountered different query languages for relational databases: Relational Algebra Tuple Relational Calculus SQL (Structured Query Language) There are, in fact, even more of them, e. g. Query by Example (QbE) Prolog/Datalog Domain Relational Calculus... Why do we care about these many query languages after all? c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 1

2 Some concepts guided the development of these query languages, e. g. (They finally lead to the SQL standard as a computer-readable query language.) Data Independence: Hide physical representation (storage on disc, etc.) from the user. Sound Mathematical Foundation: Tuple Relational Calculus is based on predicate logic. Relational Algebra uses set operations. Declarative Query Description: Describe which data we want to have, not how it should be retrieved. Example: University information system: Student (studentid, name, semester, address) Attends (studentid, courseid ) Course (courseid, title, description, ects) List Student IDs and names of all students in the Information Systems course. c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 2

3 1.1.1 SQL The Example in SQL SELECT s.studentid, s.name FROM Student s, Attends a, Course c WHERE s.studentid = a.studentid AND a.courseid = c.courseid AND c.title = Information Systems Full data independence. (No details on underlying storage.) Not fully orthogonal (in contrast to the Relational Algebra). Purely declarative. (No details on how to retrieve the data.) SQL has been designed with users in mind Hard to analyze or rewrite SQL. SQL provides tuple variables for convenience. c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 3

4 1.1.2 Tuple Relational Calculus In fact, SQL is based on the Tuple Relational Calculus {t s : Student(s) a : Attends(a) c : Course(c) s[studentid] = a[studentid] a[courseid] = c[courseid] c[title] = Information Systems t[studentid] = s[studentid] t[name] = s[name] } Full data independence. (No details on underlying storage.) Based on predicate logic. Purely declarative. (No details on how to retrieve the data.) is commutative and associative; we may put them in arbitrary order. The Tuple Relational Calculus has some real-word applications. ( list comprehensions in Haskell) c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 4

5 Haskell example: join.hs student = [ ( 4711, "John Doe", 3, "Main Street" ), ( 4712, "Marc Smith", 3, "Front Street" ), ( 4713, "Dave Meyer", 3, "West Street" ) ] attends = [ ( 4711, 12 ), ( 4711, 14 ), ( 4712, 12 ), ( 4713, 14 ) ] course = [ ( 12, "Information Systems", "Bla bla", 9 ), ( 14, "Algorithmics", "Bla bla", 9 ) ] result = [ t (s_studentid, s_name, s_semester, s_address) <- student, (a_studentid, a_courseid) <- attends, (c_courseid, c_title, c_description, c_ects_) <- course, s_studentid == a_studentid, a_courseid == c_courseid, c_title == "Information Systems", let t = (s_studentid, s_name) ] main = print result c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 5

6 1.1.3 Relational Algebra Our Example as a Relational Algebra query: π {studentid, name} σ title = Information Systems (Student Attends Course) Again, full data independence. Based on five (set) operators. Purely declarative? c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 6

7 So far we have looked at the Relational Algebra as a declarative language: Algebra operators do not specify how to implement them We did not care about things like efficiency. We ignored execution order (e. g. join order). However, the Relational Algebra is in fact a procedural language The Relational Algebra allows to define execution order. All relational DBMSs use an internal algebra representation for query optimization. The Relational Algebra is a sound and simple query specification. Optimization is achieved with help of rewrite rules. SQL is translated into an algebraic expression by the DBMS s query parser. c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 7

8 1.2 Algebraic Query Optimization Algebra expressions are usually represented as trees. ( Why?) π {studentid, name} σ title = Information Systems Course Student Attends This query plan is transformed (optimized) following rewrite rules Heuristic rules for rule-based optimization. With data statistics available, we may even do cost-based optimization. c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 8

9 1.2.1 Selection Pushdown Our original query plan generates huge intermediate results. We first fetch the student information for all courses, though we are only interested in the Information Systems course. It seems beneficial to do selections as early as possible. π {studentid, name} σ title = Information Systems π {studentid, name} Course σ title = Information Systems Student Attends Student Attends Course This is rule 14 on lecture slide c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 9

10 1.2.2 Do Projections Early The same idea holds for projections. We can reduce the size of intermediate results, if we remove unnecessary attributes as early as possible. Note that this not only reduces the size of each tuple, but may also reduce the total number of tuples. π {studentid, name} π {studentid, name} Student Attends π {courseid} σ title = Information Systems Course c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 10

11 1.2.3 Join Order can Impact Performance The left branch of our query tree computes the course attendance for all students. Only few of these will find a join partner in the second join. We d be much better off, if we picked the Course information first, then search only for attendants for this course. π {studentid} π {studentid, name} Student π {courseid} Attends σ title = Information Systems Course Note that this rewrite already requires cost estimation. c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 11

12 1.2.4 Pick Specialized Operators when Applicable The join operator is a compound operator ( Based on which operators?) In our example, we are only interested in tuples from one of the operands. The previous plan introduced projections on the join attributes for that matter. We thus introduce the semi-join as syntactic sugar: σ title = Information Systems Course Attends π {studentid, name} Student c 2004/2005 Jens Teubner Information Systems: 1. Algebraic Query Optimization 12