Copyright 2016 Ramez Elmasri and Shamkant B. Navathe

Transcription

1 Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe

2 CHAPTER 18 Strategies for Query Processig Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe

3 Itroductio DBMS techiques to process a query Scaer idetifies query tokes Parser checks the query sytax Validatio checks all attribute ad relatio ames Query tree (or query graph) created Executio strategy or query pla devised Query optimizatio Plaig a good executio strategy Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-3

4 Query Processig Figure 18.1 Typical steps whe processig a high-level query Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-4

5 18.1 Traslatig SQL Queries ito Relatioal Algebra ad Other Operators SQL Query laguage used i most RDBMSs Query decomposed ito query blocks Basic uits that ca be traslated ito the algebraic operators Cotais sigle SELECT-FROM-WHERE expressio May cotai GROUP BY ad HAVING clauses Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-5

6 Traslatig SQL Queries (cot d.) Example: Ier block Outer block Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-6

7 Traslatig SQL Queries (cot d.) Example (cot d.) Ier block traslated ito: Outer block traslated ito: Query optimizer chooses executio pla for each query block Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-7

8 Additioal Operators Semi-Joi ad Ati-Joi Semi-joi Geerally used for uestig EXISTS, IN, ad ANY subqueries Sytax: T1.X S = T2.Y T1 is the left table ad T2 is the right table of the semi-joi A row of T1 is retured as soo as T1.X fids a match with ay value of T2.Y without searchig for further matches Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-8

9 Additioal Operators Semi-Joi ad Ati-Joi (cot d.) Ati-joi Used for uestig NOT EXISTS, NOT IN, ad ALL subqueries Sytax: T1.x A = T2.y T1 is the left table ad T2 is the right table of the ati-joi A row of T1 is rejected as soo as T1.x fids a match with ay value of T2.y A row of T1 is retured oly if T1.x does ot match with ay value of T2.y Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-9

10 18.2 Algorithms for Exteral Sortig Sortig is a ofte-used algorithm i query processig Exteral sortig Algorithms suitable for large files that do ot fit etirely i mai memory Sort-merge strategy based o sortig smaller subfiles (rus) ad mergig the sorted rus Requires buffer space i mai memory DBMS cache Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-10

11 Figure 18.2 Outlie of the sort-merge algorithm for exteral sortig Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-11

12 Algorithms for Exteral Sortig (cot d.) Degree of mergig Number of sorted subfiles that ca be merged i each merge step Performace of the sort-merge algorithm Number of disk block reads ad writes before sortig is completed Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-12

13 18.3 Algorithms for SELECT Operatio SELECT operatio Search operatio to locate records i a disk file that satisfy a certai coditio File sca or idex sca (if search ivolves a idex) Search methods for simple selectio S1: Liear search (brute force algorithm) S2: Biary search S3a: Usig a primary idex S3b: Usig a hash key Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-13

14 Algorithms for SELECT Operatio (cot d.) Search methods for simple selectio (cot d.) S4: Usig a primary idex to retrieve multiple records S5: Usig a clusterig idex to retrieve multiple records S6: Usig a secodary (B+ -tree) idex o a equality compariso S7a: Usig a bitmap idex S7b: Usig a fuctioal idex Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-14

15 Algorithms for SELECT Operatio (cot d.) Search methods for cojuctive (logical AND) selectio Usig a idividual idex Usig a composite idex Itersectio of record poiters Disjuctive (logical OR) selectio Harder to process ad optimize Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-15

16 Algorithms for SELECT Operatio (cot d.) Selectivity Ratio of the umber of records (tuples) that satisfy the coditio to the total umber of records (tuples) i the file Number betwee zero (o records satisfy coditio) ad oe (all records satisfy coditio) Query optimizer receives iput from system catalog to estimate selectivity Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-16

17 18.4 Implemetig the JOIN Operatio JOIN operatio Oe of the most time cosumig i query processig EQUIJOIN (NATURAL JOIN) Two-way or multiway jois Methods for implemetig jois J1: Nested-loop joi (ested-block joi) J2: Idex-based ested-loop joi J3: Sort-merge joi J4: Partitio-hash joi Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-17

18 Implemetig the JOIN Operatio (cot d.) Figure 18.3 Implemetig JOIN, PROJECT, UNION, INTERSECTION, ad SET DIFFERENCE by usig sort-merge, where R has tuples ad S has m tuples. (a) Implemetig the operatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-18

19 Implemetig the JOIN Operatio (cot d.) Figure 18.3 (cot d.) Implemetig JOIN, PROJECT, UNION, INTERSECTION, ad SET DIFFERENCE by usig sort-merge, where R has tuples ad S has m tuples. (b) Implemetig the operatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-19

20 Implemetig the JOIN Operatio (cot d.) Figure 18.3 (cot d.) Implemetig JOIN, PROJECT, UNION, INTERSECTION, ad SET DIFFERENCE by usig sort-merge, where R has tuples ad S has m tuples. (c) Implemetig the operatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-20

21 Implemetig the JOIN Operatio (cot d.) Figure 18.3 (cot d.) Implemetig JOIN, PROJECT, UNION, INTERSECTION, ad SET DIFFERENCE by usig sort-merge, where R has tuples ad S has m tuples. (d) Implemetig the operatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-21

22 Implemetig the JOIN Operatio (cot d.) Figure 18.3 (cot d.) Implemetig JOIN, PROJECT, UNION, INTERSECTION, ad SET DIFFERENCE by usig sort-merge, where R has tuples ad S has m tuples. (e) Implemetig the operatio Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-22

23 Implemetig the JOIN Operatio (cot d.) Available buffer space has importat effect o some JOIN algorithms Nested-loop approach Read as may blocks as possible at a time ito memory from the file whose records are used for the outer loop Advatageous to use the file with fewer blocks as the outer-loop file Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-23

24 Implemetig the JOIN Operatio (cot d.) Joi selectio factor Fractio of records i oe file that will be joied with records i aother file Depeds o the particular equijoi coditio with aother file Affects joi performace Partitio-hash joi Each file is partitioed ito M partitios usig the same partitioig hash fuctio o the joi attributes Each pair of correspodig partitios is joied Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-24

25 Implemetig the JOIN Operatio (cot d.) Hybrid hash-joi Variatio of partitio hash-joi Joiig phase for oe of the partitios is icluded i the partitio Goal: joi as may records durig the partitioig phase to save cost of storig records o disk ad the rereadig durig the joiig phase Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-25

26 18.5 Algorithms for PROJECT ad Set Operatios PROJECT operatio After projectig R o oly the colums i the list of attributes, ay duplicates are removed by treatig the result strictly as a set of tuples Default for SQL queries No elimiatio of duplicates from the query result Duplicates elimiated oly if the keyword DISTINCT is icluded Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-26

27 Algorithms for PROJECT ad Set Operatios (cot d.) Set operatios UNION INTERSECTION SET DIFFERENCE CARTESIAN PRODUCT Set operatios sometimes expesive to implemet Sort-merge techique Hashig Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-27

28 Algorithms for PROJECT ad Set Operatios (cot d.) Use of ati-joi for SET DIFFERENCE EXCEPT or MINUS i SQL Example: Fid which departmets have o employees becomes Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-28

29 18.6 Implemetig Aggregate Operatios ad Differet Types of JOINs Aggregate operators MIN, MAX, COUNT, AVERAGE, SUM Ca be computed by a table sca or usig a appropriate idex Example: If a (ascedig) B+ -tree idex o Salary exists: Optimizer ca use the Salary idex to search for the largest Salary value Follow the rightmost poiter i each idex ode from the root to the rightmost leaf Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-29

30 Implemetig Aggregate Operatios ad Differet Types of JOINs (cot d.) AVERAGE or SUM Idex ca be used if it is a dese idex Computatio applied to the values i the idex Nodese idex ca be used if actual umber of records associated with each idex value is stored i each idex etry COUNT Number of values ca be computed from the idex Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-30

31 Implemetig Aggregate Operatios ad Differet Types of JOINs (cot d.) Stadard JOIN (called INNER JOIN i SQL) Variatios of jois Outer joi Left, right, ad full Example: Semi-Joi Ati-Joi No-Equi-Joi Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-31

32 18.7 Combiig Operatios Usig Pipeliig SQL query traslated ito relatioal algebra expressio Sequece of relatioal operatios Materialized evaluatio Creatig, storig, ad passig temporary results Geeral query goal: miimize the umber of temporary files Pipeliig or stream-based processig Combies several operatios ito oe Avoids writig temporary files Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-32

33 Combiig Operatios Usig Pipeliig (cot d.) Pipelied evaluatio beefits Avoidig cost ad time delay associated with writig itermediate results to disk Beig able to start geeratig results as quickly as possible Iterator Operatio implemeted i such a way that it outputs oe tuple at a time May iterators may be active at oe time Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-33

34 Combiig Operatios Usig Pipeliig (cot d.) Iterator iterface methods Ope() Get_Next() Close() Some physical operators may ot led themselves to the iterator iterface cocept Pipeliig ot supported Iterator cocept ca also be applied to access methods Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-34

35 18.8 Parallel Algorithms for Query Processig Parallel database architecture approaches Shared-memory architecture Multiple processors ca access commo mai memory regio Shared-disk architecture Every processor has its ow memory Machies have access to all disks Shared-othig architecture Each processor has ow memory ad disk storage Most commoly used i parallel database systems Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-35

36 Parallel Algorithms for Query Processig (cot d.) Liear speed-up Liear reductio i time take for operatios Liear scale-up Costat sustaied performace by icreasig the umber of processors ad disks Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-36

37 Parallel Algorithms for Query Processig (cot d.) Operator-level parallelism Horizotal partitioig Sortig Roud-robi partitioig Rage partitioig Hash partitioig If data has bee rage-partitioed o a attribute: Each partitio ca be sorted separately i parallel Results cocateated Reduces sortig time Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-37

38 Parallel Algorithms for Query Processig (cot d.) Selectio If coditio is a equality coditio o a attribute used for rage partitioig: Perform selectio oly o partitio to which the value belogs Projectio without duplicate elimiatio Perform operatio i parallel as data is read Duplicate elimiatio Sort tuples ad discard duplicates Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-38

39 Parallel Algorithms for Query Processig (cot d.) Parallel jois divide the joi ito smaller jois Perform smaller jois i parallel o processors Take a uio of the result Parallel joi techiques Equality-based partitioed joi Iequality joi with partitioig ad replicatio Parallel partitioed hash joi Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-39

40 Parallel Algorithms for Query Processig (cot d.) Aggregatio Achieved by partitioig o the groupig attribute ad the computig the aggregate fuctio locally at each processor Set operatios If argumet relatios are partitioed usig the same hash fuctio, they ca be doe i parallel o each processor Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-40

41 Parallel Algorithms for Query Processig (cot d.) Itraquery parallelism Approaches Use parallel algorithm for each operatio, with appropriate partitioig of the data iput to that operatio Execute idepedet operatios i parallel Iterquery parallelism Executio of multiple queries i parallel Goal: scale up Difficult to achieve o shared-disk or sharedothig architectures Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-41

42 18.9 Summary SQL queries traslated ito relatioal algebra Exteral sortig Selectio algorithms Joi operatios Combiig operatios to create pipelied executio Parallel database system architectures Copyright 2016 Ramez Elmasri ad Shamkat B. Navathe Slide 18-42