Copyright 2016 Ramez Elmasri and Shamkant B. Navathe

Transcription

1 Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe

2 CHAPTER 19 Query Optimizatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe

3 Itroductio Query optimizatio Coducted by a query optimizer i a DBMS Goal: select best available strategy for executig query Based o iformatio available Most RDBMSs use a tree as the iteral represetatio of a query Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-3

4 19.1 Query Trees ad Heuristics for Query Optimizatio Step 1: scaer ad parser geerate iitial query represetatio Step 2: represetatio is optimized accordig to heuristic rules Step 3: query executio pla is developed Execute groups of operatios based o access paths available ad files ivolved Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-4

5 Query Trees ad Heuristics for Query Optimizatio (cot d.) Example heuristic rule Apply SELECT ad PROJECT before JOIN Reduces size of files to be joied Query tree Represets relatioal algebra expressio Query graph Represets relatioal calculus expressio Example for Q2 o ext slide Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-5

6 Query Trees ad Query Graph Correspodig to Q2 Figure 19.1 Two query trees for the query Q2. (a) Query tree correspodig to the relatioal algebra expressio for Q2. (b) Iitial (caoical) query tree for SQL query Q2. (c) Query graph for Q2. Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-6

7 Query Trees ad Heuristics for Query Optimizatio (cot d.) Query tree represets a specific order of operatios for executig a query Preferred to query graph for this reaso Query graph Relatio odes displayed as sigle circles Costats represeted by costat odes Double circles or ovals Selectio or joi coditios represeted as edges Attributes to be retrieved displayed i square brackets Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-7

8 Heuristic Optimizatio of Query Trees May differet query trees ca be used to represet the query ad get the same results Figure 19.1b shows iitial tree for Q2 Very iefficiet - will ever be executed Optimizer will trasform ito equivalet fial query tree Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-8

9 Query Trasformatio Example Figure 19.2 Steps i covertig a query tree durig heuristic optimizatio. (a) Iitial (caoical) query tree for SQL query Q. Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-9

10 Query Trasformatio Example (cot d.) Figure 19.2 Steps i covertig a query tree durig heuristic optimizatio (b) Movig SELECT operatios dow the query tree. Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-10

11 Query Trasformatio Example (cot d.) Figure 19.2 Steps i covertig a query tree durig heuristic optimizatio (c) Applyig the more restrictive SELECT operatio first. Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-11

12 Query Trasformatio Example (cot d.) Figure 19.2 Steps i covertig a query tree durig heuristic optimizatio (d) Replacig CARTESIAN PRODUCT ad SELECT with JOIN operatios. Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-12

13 Query Trasformatio Example (cot d.) Figure 19.2 Steps i covertig a query tree durig heuristic optimizatio (e) Movig PROJECT operatios dow the query tree. Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-13

14 Geeral Trasformatio Rules for Ratioal Algebra Equatios Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-14

15 Summary of Heuristics for Algebraic Optimizatio Apply first the operatios that reduce the size of itermediate results Perform SELECT ad PROJECT operatios as early as possible to reduce the umber of tuples ad attributes The SELECT ad JOIN operatios that are most restrictive should be executed before other similar operatios Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-15

16 19.2 Choice of Query Executio Plas Materialized evaluatio Result of a operatio stored as temporary relatio Pipelied evaluatio Operatio results forwarded directly to the ext operatio i the query sequece Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-16

17 Nested Subquery Optimizatio Uestig Process of removig the ested query ad covertig the ier ad outer query ito oe block Queries ivolvig a ested subquery coected by IN or ANY coector ca always be coverted ito a sigle block query Alterate techique Creatig temporary result tables from subqueries ad usig them i jois Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-17

18 Subquery (View) Mergig Trasformatio Ilie view FROM clause subquery View mergig operatio Merges the tables i the view with the tables from the outer query block Views cotaiig select-project-joi operatios are cosidered simple views Ca always be subjected to this type of viewmergig Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-18

19 Subquery (View) Mergig Trasformatio (cot d.) Group-By view-mergig Delayig the Group By operatio after performig jois may reduce the data subjected to groupig i case the jois have low joi selectivity Alterately, performig Group By early may reduce the amout of data subjected to subsequet jois Optimizer determies whether to merge GROUP- BY views based o estimated costs Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-19

20 Materialized Views View defied i database as a query Materialized view stores results of that query May be stored temporarily or permaetly Optimizatio techique Usig materialized views to avoid some of the computatio ivolved i the query Easier to read it whe eeded tha recompute from scratch Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-20

21 Icremetal View Maiteace Update view icremetally by accoutig for chages that occurred sice last update Joi Selectio Projectio Itersectio Aggregatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-21

22 19.3 Use of Selectives i Cost-Based Optimizatio Query optimizer estimates ad compares costs of query executio usig differet strategies Chooses lowest cost estimate strategy Process suited to compiled queries Iterpreted queries Etire process occurs at rutime Cost estimate may slow dow respose time Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-22

23 Use of Selectives i Cost-Based Optimizatio (cot d.) Cost-based query optimizatio approach For a give query subexpressio, multiple equivalece rules may apply Quatitative measure for evaluatig alteratives Cost metric icludes space ad time requiremets Desig appropriate search strategies by keepig cheapest alteratives ad pruig costlier alteratives Scope of query optimizatio is a query block Global query optimizatio ivolves multiple query blocks Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-23

24 Use of Selectives i Cost-Based Optimizatio (cot d.) Cost compoets for query executio Access cost to secodary storage Disk storage cost Computatio cost Memory usage cost Commuicatio cost Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-24

25 Catalog Iformatio Used i Cost Fuctios Iformatio stored i DBMS catalog ad used by optimizer File size Orgaizatio Number of levels of each multilevel idex Number of distict values of a attribute Attribute selectivity Allows calculatio of selectio cardiality Average umber of records that satisfy equality selectio coditio o that attribute Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-25

26 Histograms Tables or data structures that record iformatio about the distributio of data RDBMS stores histograms for most importat attributes Figure 19.4 Histogram of salary i the relatio EMPLOYEE Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-26

27 19.4 Cost Fuctios for SELECT Operatio Notatio used i cost formulas Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-27

28 Cost Fuctio for SELECT Operatio (cot d.) S1: Liear search (brute force approach) Search all file blocks to retrieve all records C S1a =b For equality coditio o a key attribute, o average oe-half the records are searched C S1b =! " S2: Biary search C S2 =log 2 b+[ #!$% ]-1 Reduces to log 2 b if equality coditio is o a key attribute Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-28

29 Cost Fuctio for SELECT Operatio (cot d.) S3a: Usig a primary idex to retrieve a sigle record C S3a = x + 1 S3b: Usig a hash key to retrieve a sigle record C S3b = 1 S4: Usig a orderig idex to retrieve multiple records C S4 = x +! " Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-29

30 Cost Fuctios for SELECT Operatio (cot d.) Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-30

31 Cost Fuctios for SELECT Operatio (cot d.) Dyamic programmig Cost-based optimizatio approach Subproblems are solved oly oce Applies whe a problem has subproblems that themselves have subproblems Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-31

32 19.5 Cost Fuctios for the JOIN Operatio Cost fuctios ivolve estimate of file size that results from the JOIN operatio Joi selectivity Ratio of the size of resultig file to size of the CARTESIAN PRODUCT file Simple formula for joi selectivity Joi cardiality Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-32

33 Cost Fuctios for the JOIN Operatio (cot d.) J1: Nested-loop joi For three memory buffer blocks: For B memory buffer blocks: J2: Idex-based ested-loop joi For a secodary idex with selectio cardiality S B for joi attribute B of S: Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-33

34 Cost Fuctios for the JOIN Operatio (cot d.) J3: Sort-merge joi For files already sorted o the joi attributes Cost of sortig must be added if sortig eeded J4: Partitio-hash joi Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-34

35 Cost Fuctios for the JOIN Operatio (cot d.) Joi selectivity ad cardiality for semi-joi Uestig query above leads to semi-joi Joi selectivity Joi cardiality Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-35

36 Cost Fuctios for the JOIN Operatio (cot d.) Joi selectivity ad cardiality for ati-joi Uestig query above leads to ati-joi Joi selectivity Joi cardiality Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-36

37 Cost Fuctios for the JOIN Operatio (cot d.) Multirelatio queries ad JOIN orderig choices Left-deep joi tree Right-deep joi tree Bushy joi tree Table 19.1 Number of permutatios of left-deep ad bushy joi trees of relatios Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-37

38 Cost Fuctios for the JOIN Operatio (cot d.) Physical optimizatio ivolves executio decisio at the physical level Cost-based physical optimizatio Top-dow approach Bottom-up approach Certai physical level heuristics make cost optimizatios uecessary Example: for selectios, use idex scas wheever possible Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-38

39 Cost Fuctios for the JOIN Operatio (cot d.) Left-deep trees geerally preferred Work well for commo algorithms for joi Able to geerate fully pipelied plas Characteristics of dyamic programmig algorithm Optimal solutio structure is developed Value of the optimal solutio is recursively defied Optimal solutio is computed ad its value developed i a bottom-up fashio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-39

40 19.6 Example to Illustrate Cost-Based Query Optimizatio Example: Cosider Q2 below ad query tree from Figure 19.1(a) o slide 6 Iformatio about the relatios show i Figure 19.6 (ext slide) Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-40

41 Figure 19.6 Sample statistical iformatio for relatios i Q2. (a) Colum iformatio (b) Table iformatio (c) Idex iformatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-41

42 Example to Illustrate Cost-Based Query Optimizatio (cot d.) Assume optimizer cosiders oly left-deep trees Evaluate potetial joi orders PROJECT DEPARTMENT EMPLOYEE DEPARTMENT PROJECT EMPLOYEE DEPARTMENT EMPLOYEE PROJECT EMPLOYEE DEPARTMENT PROJECT Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 16-42

43 19.7 Additioal Issues Related to Query Optimizatio Displayig the system s query executio pla Oracle sytax EXPLAIN PLAN FOR <SQL query> IBM DB2 sytax EXPLAIN PLAN SELECTION [additioal optios] FOR <SQL-query> SQL server sytax SET SHOWPLAN_TEXT ON or SET SHOWPLAN_XML ON or SET SHOWPLAN_ALL ON Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-43

44 Additioal Issues Related to Query Optimizatio (cot d.) Size estimatio of other operatios Projectio Set operatios Aggregatio Outer joi Pla cachig Pla stored by query optimizer for later use by same queries with differet parameters Top k-results optimizatio Limits strategy geeratio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-44

45 19.8 A Example of Query Optimizatio i Data Warehouses Star trasformatio optimizatio Goal: access a reduced set of data from the fact table ad avoid usig a full table sca o it Classic star trasformatio Bitmap idex star trasformatio Joiig back Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-45

46 19.9 Overview of Query Optimizatio i Oracle Physical optimizer is cost-based Scope is a sigle query block Calculates cost based o object statistics, estimated resource use ad memory eeded Global query optimizer Itegrates logical trasformatio ad physical optimizatio phases to geerate optimal pla for etire query tree Adaptive optimizatio Feedback loop used to improve o previous decisios Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-46

47 Overview of Query Optimizatio i Oracle (cot d.) Array processig Hits Specified by applicatio developer Embedded i text of SQL statemet Types: access path, joi order, joi method, eablig or disablig a trasformatio Outlies used to preserve executio plas SQL pla maagemet Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-47

48 19.10 Sematic Query Optimizatio Uses costraits specified o the database schema Goal: modify oe query ito aother that is more efficiet to execute Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-48

49 19.11 Summary Query trees Heuristic approaches used to improve efficiecy of query executio Reorgaizatio of query trees Pipeliig ad materialized evaluatio Cost-based optimizatio approach Oracle query optimizer Sematic query optimizatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 19-49