QUERY PROCESSING & OPTIMIZATION CHAPTER 19 (6/E) CHAPTER 15 (5/E)

Transcription

1 QUERY PROCESSING & OPTIMIZATION CHAPTER 19 (6/E) CHAPTER 15 (5/E)

2 2 LECTURE OUTLINE Query Processing Methodology Basic Operations and Their Costs Generation of Execution Plans

3 3 QUERY PROCESSING IN A DDBMS high level user query query processor Low-level data manipulation commands for D-DBMS

4 4 SELECT OPERATOR Unary operator (one relation as operand) Returns subset of the tuples from a relation that satisfies a selection condition: σ <selection condition> (R) where <selection condition> may have Boolean conditions AND, OR, and NOT has clauses of the form: <attribute name> <comparison op> <constant value> or <attribute name> <comparison op> <attribute name> Applied independently to each individual tuple t in operand Tuple selected iff condition evaluates to TRUE Example: σ (Dno = 4 AND Salary>2500) OR (Dno=5 AND Salary>30000) (EMPLOYEE)

5 5 SELECT OPERATOR (CONT D.) Do not confuse this with SQL s SELECT statement! Correspondence Relational algebra σ <selection condition> (R) SQL SELECT * FROM R WHERE <selection condition> Equivalences Selectivity Fraction of tuples selected by a selection condition

6 6 PROJECT OPERATOR Unary operator (one relation as operand) Keeps specified attributes and discards the others: Duplicate elimination Result of PROJECT operation is a set of distinct tuples Example: Correspondence Relational algebra SQL SELECT DISTINCT <attribute list> FROM R Note the need for DISTINCT in SQL

7 7 PROJECT OPERATOR PROPERTIES π L (R) is defined only when L attr (R ) Equivalences as long as all attributes used by C are in L Degree Number of attributes in projected attribute list

8 8 CROSS PRODUCT OPERATOR Binary operator aka CARTESIAN PRODUCT or CROSS JOIN R S Attributes of result is union of attributes in operands deg(r S) = deg(r) + deg(s) Tuples in result are all combinations of tuples in operands R S = R * S Relations do not have to be union compatible Often followed by a selection that matches values of attributes What if both operands have an attribute with the same name?

9 9 RENAMING RELATIONS & ATTRIBUTES Unary RENAME operator Rename relation Rename attributes Rename relation and its attributes Student name year Ashley 4 Lee 3 Dana 1 Jo 1 Jaden 2 Billie 3 Example: pairing upper year students with freshmen

10 10 JOIN OPERATOR Binary operator where join condition is a Boolean expression involving attributes from both operand relations Like cross product, combine tuples from two relations into single longer tuples, but only those that satisfy matching condition Formally, a combination of cross product and select aka θ-join or inner join Join condition expressed as A θ B, where θ {=,,>,,<, } as opposed to outer joins, which will be explained later Join selectivity Fraction of number tuples in result over maximum possible Common case (as in examples above): equijoin

11 11 REVIEW OF OPERATORS Select Project Rename Union Intersection Difference Cross product Join Natural join

12 12 SELECTING ALTERNATIVES SELECT FROM WHERE AND ENAME EMP,ASG EMP.ENO = ASG.ENO ASG.RESP = "Manager" Strategy 1 ENAME ( RESP= Manager EMP.ENO=ASG.ENO (EMP ASG)) Strategy 2 ENAME (EMP ENO ( RESP= Manager (ASG)) Strategy 2 avoids Cartesian product, so may be better

13 13 PICTORIALLY Strategy 1 Strategy 2 ENAME ENAME ASG.RESP= Manager EMP.ENO=ASG.ENO ENO ASG.RESP= Manager EMP ASG EMP ASG

14 14 QUERY PROCESSING METHODOLOGY generate alternative access plans, i.e., procedure, for processing the query select an efficient access plan Query in high-level language Query Decomposition & Translation Algebraic Query Query Optimization Query Execution Plan SQL check SQL syntax check existence of relations and attributes replace views by their definitions transform query into an internal form Query Code Generator Code to execute query Runtime Processor Query Result

15 15 EXAMPLE SELECT V.Vno, Vname, count(*), sum(amount) FROM Vendor V, Transaction T WHERE V.Vno = T.Vno AND V.Vno between 1000 and 2000 GROUP BY V.Vno, Vname HAVING sum(amount) > 100 Scan the Vendor table, select all tuples where Vno = [1000, 2000], eliminate attributes other than Vno and Vname, and place the result in a temporary relation R 1 Join the tables R 1 and Transaction, eliminate attributes other than Vno, Vname, and Amount, and place the result in a temporary relation R 2. This may involve: sorting R 1 on Vno sorting Transaction on Vno merging the two sorted relations to produce R 2 Perform grouping on R 2, and place the result in a temporary relation R 3. This may involve: sorting R 2 on Vno and Vname grouping tuples with identical values of Vno and Vname counting the number of tuples in each group, and adding their Amounts Scan R 3, select all tuples with sum(amount) > 100 to produce the result.

16 16 EXAMPLE SELECT V.Vno, Vname, count(*), sum(amount) FROM Vendor V, Transaction T WHERE V.Vno = T.Vno AND 2000 V.Vno between 1000 and GROUP BY V.Vno, Vname HAVING sum(amount) > 100 Scan (Sum(Amount) > 100) Grouping (Vno, Vname) Join (V.Vno = T.Vno) Scan (Vno between 1000 and 2000) Vendor Transaction

17 17 QUERY OPTIMIZATION ISSUES Determining the shape of the execution plan Order of execution Determining which how each node in the plan should be executed Operator implementations These are interdependent and an optimizer would do both in generating the execution plan

18 18 SHAPE OF THE EXECUTION PLAN Finding query trees that are equivalent Produce the same result provably These are based on the transformation (equivalence) rules Commutativity of selection p1 (A 1 )( p2 (A 2 )R) p2 (A 2 )( p1 (A 1 )R) Commutativity of binary operations R S S R R S S R R S S R R S S R Associativity of binary operations ( R S) T R (S T) (R S) T R (S T) (R S) T (S R) T Cascading of unary operations A ( A (R)) A (R) where R[A] and A' A, A" A and A' A" p1 (A 1 )( p2 (A 2 )(R)) p1 (A 1 ) p 2 (A 2 )(R)

19 19 OTHER TRANSFORMATION RULES Commuting selection with projection B ( p(a) R) p(a) ( B R) (where B A) Commuting selection with binary operations p(a) (R S) ( p(a) (R)) S (where A belongs to R only) p(ai )(R (Aj,B k )S) ( p(ai ) (R)) (Aj,B k )S (where A i belongs to R only) p(ai )(R S) p(ai ) (R) p(ai ) (S) (where A i belongs to R and S) p(ai )(R S) p(ai ) (R) p(ai ) (s) (where A i belongs to R and S) Commuting projection with binary operations C (R S) A (R) B (S) C (R (Aj,B k )S) A (R) (Aj,B k ) B (S) C (R S) C (R) C (S) C (R S) C (R) C (S) where R[A] and S[B]; C = A' B' where A' A, B' B

20 20 EXAMPLE TRANSFORMATION Find the names of employees other than J. Doe who worked on the CAD/CAM project for either one or two years. E.ENAME G.DUR=12 G.DUR=24 Project SELECT ENAME FROM PROJ P, ASG G, EMP E WHERE G.ENO=E.ENO AND G.PNO=P.PNO AND E.ENAME <> 'J. Doe' AND P.PNAME='CAD/CAM' AND (G.DUR=12 OR G.DUR=24) P.PNAME= CAD/CAM E.ENAME<> J. DOE PNO ENO Select Join PROJ ASG EMP

21 21 EQUIVALENT QUERY E.ENAME P.PNAME= CAD/CAM (G.DUR=12 G.DUR=24) E.ENAME<> J. Doe PNO,ENO EMP PROJ ASG

22 22 ANOTHER EQUIVALENT QUERY ENAME PNO PNO,ENAME EN O PNO PNO,ENO PNO,ENAME PNAME = "CAD/CAM" DUR =12 DUR=24 ENAME "J. Doe" PROJ ASG EMP

23 23 QUESTION Is the right query plan equivalent to the left query plan? E.ENAME G.DUR=12 G.DUR=24 E.ENAME P.PNAME= CAD/CAM EN O E.ENAME<> J. DOE PNO PNO EN O P.PNAME= CAD/CAM G.DUR=12 G.DUR=24 E.ENAME<> J. DOE PROJ ASG EMP PROJ ASG EMP (a) Yes (b) No

24 24 IMPORTANT PROBLEM JOIN ORDER Assume you have R S T W Linear Join Tree W Bushy Join Tree T R S R S T T Most systems implement linear join trees Left-linear

25 25 JOIN ORDERING Even with left-linear, how do you know which order? Assume natural join over common attributes W T T T W W R S R S R S

26 26 SOME OPERATOR IMPLEMENTATIONS Tuple Selection without an index with a clustered index with an unclustered index with multiple indices Projection Joining nested loop join sort-merge join and others... Grouping and Duplicate Elimination by sorting by hashing Sorting

27 27 EXAMPLE JOIN ALGORITHMS SELECT C.Cnum, A.Balance FROM Customer C, Accounts A WHERE C.Cnum = A.Cnum Nested loop join: for each tuple c in Customer do for each tuple a in Accounts do if c.cnum = a.cnum then output c.cnum,a.balance end end

28 28 EXAMPLE JOIN ALGORITHMS (2) SELECT FROM WHERE C.Cnum, A.Balance Customer C, Accounts A C.Cnum = A.Cnum Index join: for each tuple c in Customer do use the index to find Accounts tuples a where a.cnum matches c.cnum if there are any such tuples a then output c.cnum, a.balance end end Sort-merge join: sort Customer and Accounts on Cnum merge the resulting sorted relations

29 29 COMPLEXITY OF OPERATORS Assume Relations of cardinality n Sequential scan Operation Complexity Select Project (without duplicate elimination) Project (with duplicate elimination) Group Join Semi-join Division Set Operators O(n) O(n log n) O(n log n) Cartesian Product O(n 2 )

30 30 COST OF PLANS Alternative access plans may be compared according to cost. The cost of an access plan is the sum of the costs of its component operations. There are many possible cost metrics. However, most metrics reflect the amounts of system resources consumed by the access plan. System resources may include: disk block I/O s processing time network bandwidth

31 31 LECTURE SUMMARY Query processing methodology Basic query operations and their costs Generation of execution plans