Chapter 2 Introduction to Query Optimization

Transcription

1 4 Chapter 2 ntroducton to Query Optmzaton 2.1 Query Optmzaton Outlne n ths secton, we wll cover the followng topcs:. 1) Query Processor Components tokenzer, parser, access planner, optmzer, buffer manager logcal and physcal query plans. 2) Query Optmzaton relatonal algebra laws for query plans cost based optmzaton versus heurstc optmzaton 2.2 Query Optmzaton Overvew The goal of the query processor s very smple: Return the answer to an SQL query n the most effcent way possble gven the fle organzaton of the database. However, actually achevng ths goal s anythng but smple:. Dfferent fle organzatons and ndexng affect performance.. Dfferent algorthms can be used to perform the relatonal algebra operatons wth varyng performance based on the DB.. Estmatng the cost of the query tself s hard.. Determnng the best way to answer one query n solaton s challengng, how about many concurrent queres? Components of a Query Processor db stats SQL Query L Parser Expresson Tree Translator T Logcal Query Tree Optmzer. l physcal Query Tree Evaluator ^ Query Output SELECT name PROM Student WHERE Major=*CS" sblbct <SelLst> <Attr» uatn ^ ^ Mmser (7,Mt#or-=TCS- Student <7romLl&t> < Condton;. / / / «Rpl> <A.ttr»- <value> Student -**cr. Major CS n V'wnc (tabla aeon) ~ ( ndex ftcaol Student Page 4

2 5 Query Processor Components The Parser The role of the parser s to convert a SQL statement represented as a strng of characters nto a parse tree. A parse tree conssts of nodes, and each node s ether an:. Atom - lexcal elements such as words (WHERE), attrbute or relaton names, constants, operator symbols, etc.. Syntactc category - are names for query subparts..e.g. <SFW> represents a query n select-from-where form. Nodes that are atoms have no chldren. Nodes that correspond to categores have chldren based on one of the rules of the grammar for the language A Smple SQL Grammar A grammar s a set of rules dctatng the structure of the language. t exactly specfes what strngs correspond to the language and what ones do not. Complers are used to parse grammars nto parse trees. Same process for SQL as programmng languages, but somewhat smpler because the grammar for SQL s smaller. Our smple SQL grammar wll only allow queres n the form of SELECT-FROM-WHERE. We wll not support groupng, orderng, or SELECT DSTNCT. We wll have to support lsts of attrbutes n the SELECT clause, lsts of relatons n the FROM clause, and condtons n the WHERE clause. Smple SQL Grammar <Query> ::= <SFW> <Query> ::= (<Query> ) <SFW> ::= SELECT <SelLst> FROM <FromLst> WHERE <Condton> <SelLst> ::= <Atn> <SelLst> <Attr>, <SeLst> <FromLst> ::= <Rel> <FromLst> ::= <Rel>, <FromLst> <Condton> ::= <Condton> AND <Condton> <Condton> ::= <Tuple> N <Query> <Condton> ::= <Att> = <Att> <Condton> ::= <Att> LKE <Value> <Condton> ::= <Attr> = <Value> <Tuple> ::= <Attr> //Tuple may be 1 attrbute A Smple SQL Grammar Dscusson The syntactc categores of <Attr>, <Rel>, and <Value> are specal because they are not defned by the rules of the grammar.. <Attr> - must be a strng of characters that matches an attrbute name n the database schema., <Rel> - must be a character strng that matches a relaton name n the database schema, <Value> - s some quoted strng that s a legal SQL pattern or a vald numercal value.

3 Query Example Database Student (d,name,major,year) Department (Code, Dept Name, Locaton) Student Relaton St D Name Mjr Yr Doe CS T. Allen BA M. Smth CS B, Zmmer BS T. Atkns ME J.Wona BA S. Allen CS P. Wrsht ME 2 Department Relaton Code DeptName Locaton BA Bachelor of Englsh Arts Buldna BS Bachelor of Physcs Scence Buldna CS Computer Scence MacLean Hall ME Mechancal Eneneernc Engneerng Buldna Query Parsng Example Return all students who major n computer scence. SELECT Name PROM Sttulent WHERE Major="CS** <Qu^ry> SELECT <3eu.la t> <Attr> HaLe FROM 1 * KHERB <FrofLl8t> <Codad±tlon> / / / l <B^l> Student <Attr>- <Value > *ds«major Rule a appled; <Query» ::= <SFW» <SPW> : := SEtBCT <Salst> FROM «PromLst» WHER8 «Condton> <SelLlat> : : = <Attr> <<Attr> m ' Name ) <Condtl on> : := <Attr> = <Value> (<Attr>* Major", <Valus>=*CS" ) <ProraLst> ::= <Rel> (<Rel> = Student") PagetO Query Parsng Example 2 Return all departments who have a 4th year student. SELECT DeptName FROM Department, Student WHERE Code - Major AND Year «4 4 ary» 9B1BCT <SalllBt^ <Attr> >tlram Dept <Rel> Department <rranfl.lst> WHERE <Cmmlton> s / \ <Froan.st> <Co^ay:an> AMD <Ctm«nton» <s>l> «Attr»- «Attr> <Attr>- <value> r j Student Code Major Year 4 can you determne wlat rules are appled?

4 7 Query Parsng Example 3 Return all departments who have a 4th year student. 4 SELECT DeptName FROM Department WHERE Code N (SELECT Major FROM Student WHERE Year=4) SELECT <selllat^ <Attr> Deptjlama <Que ery> «Con<lltlcn» / / <Tuplo> k <Qu«cy> <T,romLlet> WHERE f <Attr> Department SELECT <SalLlst> <Canalt±on-» <VaLue> Tear Query Processor Components The Parser Functonalty The parser converts an SQL strng to a parse tree. Ths nvolves breakng the strng nto tokens. Each token s matched wth the grammar rules accordng to the current parse tree. nvald tokens (not n grammar) generate an error. f there are no rules n the grammar that apply to the current SQL strng, the command wll be flagged to have a syntax error. We wll not concern ourselves wth how the parser works. However, we wll note that the parser s responsble for checkng for syntax errors n the SQL statement. That s, the parser determnes f the SQL statement s vald accordng to the grammar Query Processor Components The Preprocessor The preprocessor s a component of the parser that performs semantc valdaton. The preprocessor runs after the parser has bult the parse tree. ts functons nclude: Mappng vews nto the parse tree f requred.. Verfy that the relaton and attrbute names are actually vald relatons and attrbutes n the database schema.. Verfy that attrbute names have a correspondng relaton name specfed n the query. (Resolve attrbute names to relatons.). Check types when comparng wth constants or other attrbutes. f a parse tree passes syntax and semantc valdaton, t s called a vald parse tree. A vald parse tree s sent to the logcal query processor, otherwse an error s sent back to the user.

5 Query Processor Components Translator The translator, or logcal query processor, s the component that takes the parse tree and converts t nto a logcal query tree. A logcal query tree s a tree consstng of relatonal operators and relatons. t specfes what operatons to apply, and the order to apply them, but not how to actually mplement the operatons. A logcal query tree does not select a partcular algorthm to mplement each relatonal operator. We wll gve some nformal rules explanng how the parse tree s converted nto a logcal query tree Convertng Parse Trees to Logcal Query Trees The smplest parse tree to convert s one where there s only one select-from-where (<SFW>) construct, and the <Condton> construct has no nested queres. The logcal query tree produced conssts of:. 1) The cross-product of all relatons mentoned n the <FromLst> whch are nputs to:. 2) A selecton operator, c, where C s the <Condton> expresson n the construct beng replaced whch s the nput to:, 3) A projecton, %, where L s the lst of attrbutes n the <SelLst>. Parse Tree to Logcal Tree Example SELECT Name FROM Student WHERE Major=*CS" <Ouery> 8 SELECT <SeXUst> «Atlr> Rate <Oon<ll t±on> / / / <Attr»- <Value> ds* student Hal or n Earner a X&QorssTCS Student Parse Tree to Logcal Tree Example 2 SELECT DeptName FROM Department, Student WHERE Code = Major AND Year = 4

6 Convertng Nested Parse Trees to Logcal Query Trees Convertng a parse tree that contans a nested query s slghtly more challengng. A nested query may be correlated wth the outsde query f t must be re-computed for every tuple produced by the outsde query. Otherwse, t s uncorrelated, and the nested query can be converted to a non-nested query usng jons. We wll defne a two-operand selecton operator Dthat takes the outer relaton R as one nput (left chld), and the rght chld s the condton appled to each tuple of R. The condton s the subquery nvolvng n. Convertng Nested Parse Trees to Logcal Query Trees (2) The nested subquery translaton algorthm nvolves defnng a tree from root to leaves as follows:, 1) Root node s a projecton, *l, where L s the lst of attrbutes n the <SelLst> of the outer query.. 2) Chld of root s a selecton operator, c-, where C s the <Condton> expresson n the outer query gnorng the subquery.. 3) The two-operand selecton operator wth left-chld as the cross-product (x) of all relatons mentoned n the <FromLst> of the outer query, and rght chld as the <Condton> expresson for the subquery.. 4) The subquery tself nvolved n the <Condton> expresson s translated to relatonal algebra. Parse Tree to Logcal Tree Example 3 SELECT DeptName FROM Department WHERE Code N (SELECT Major FROM Student WHERE Year=4)

7 10 Parse Tree to Logcal Tree Example 3 (2) select Deptwam from Deparcmenc where code n (SELECT Major PROM Student WHERE Year=4) n Dcp&'amc^ No outer level selecton. X Only one outer e} Department <oan<uton>^9 Condton n parse tree, relaton. '' ' L <Tuple> <Attr> Code n cr. student Subquery translated to loycal query tree Convertng Nested Parse Trees to Logcal Query Trees (3) Now, we must remove the two-operand selecton and replace t by relatonal algebra operators. Rule for replacng two-operand selecton (uncorrelated): Let R be the frst operand, and the second operand s a <Condton> of the form t N S. (S s uncorrelated subquery.) 1) Replace <Condton> by the tree that s expresson for 5..May requre applyng duplcate elmnaton f expresson has duplcates. 2) Replace two-operand selecton by one-argument selecton, c, where C s the condton that equates each component of the tuple t to the correspondng attrbute of relaton S. 3) Gve c an argument that s the product of R and S. Parse Tree to Logcal Tree Converson Replaced C wth Oc and X, May need to 5 ^3 elmnate duplcates A 2.4 Query Optmzaton Process Rules of Query Optmzaton 1. Deconstruct conjunctve selectons nto a sequence of sngle selecton operatons. 2. Move selecton operatons down the query tree for the earlest possble executon. 3. Execute frst those selecton and jon operatons that wll produce the smallest relatons. 4. Replace Cartesan product operatons that are followed by a selecton condton by jon operatons.

8 11 5. Deconstruct and move as far down the tree as possble lsts of projecton attrbutes, creatng new projectons where needed. 6. dentfy those subtrees whose operatons can be ppelned, and execute them usng ppelnng. Heurstc Optmzaton Example 2 SELECT DeptName FROM Department, Student WHERE Code a Major AND Year» 4 (T, S tu de.n t. Orfoln al: v.- =.'> f-j] --.w \N1) >...r- l L Department Optmzatons - push selecton down - push projecton down merge selecton and cross-product *l'deptnsrat>(t^rafc-,wq^r AMD -etr- A NVU/crlt X t)l'po'ullchl)) Optmzed: ^DepKatteU (Ty^^StUccnt)) **Cak-Mqfar * Student ^DtjuNaac O^J Major=Code *Dq)tNaro»,Colc. Department Physcal Query Plan n Query Optmzaton A physcal query plan s derved from a logcal query plan by: 1) Selectng an order and groupng for operatons lke jons,unons, and ntersectons. 2) Decdng on an algorthm for each operator n the logcal query plan. e.g. Nested-loop jon, sort jon or hash jon 3) Addng addtonal operators to the logcal query tree such as sortng and scannng that are not present n the logcal plan. 4) Determnng how arguments are passed from one operator to the next. nvolves decdng between ppelnng and materalzaton. Whether we perform cost-based or heurstc optmzaton, we eventually must arrve at a physcal query tree that can be executed by the evaluator. Query Optmzaton Heurstc versus Cost Optmzaton n order to determne when one physcal query plan s better than another, we must have an estmate of the cost of the plan. Heurstc optmzaton s normally used to pck the best logcal query plan. Cost-based optmzaton s used to determne the best physcal query plan gven a logcal query plan. Note that both can be used n the same query processor (and typcally are). Heurstc optmzaton s used to pck the best logcal plan whch s then optmzed by cost-based technques. 2.5 Query Optmzaton Estmatons Estmatng Operaton Cost n order to determne when one physcal query plan s better than another for cost-based optmzaton, we must have an estmate of the cost of a physcal query plan. Note that the

9 12 query optmzer wll very rarely know the exact cost of a query plan because the only way to know s to execute the query tself! Snce the cost to execute a query s much greater than the cost to optmze a query, we cannot execute the query to determne ts cost! Thus, t s mportant to be able to estmate the cost of a query plan wthout executng t based on statstcs and general formulas Estmatng Projecton Szes Calculatng the sze of a relaton after the projecton operaton s easy because we can compute t drectly. Assumng we know the sze of the nput, we can calculate the sze of the output based on the sze of the nput records and the sze of the output records. The projecton operator decreases the sze of the tuples, not the number of tuples. For example, gven relaton R(a,b,c) wth sze of a = sze of b = 4 bytes, and sze of c = 100 bytes. T(R) = and unspanned block sze s 1024 bytes. f the projecton operaton s *\t», what s the sze of the output U n blocks? T(U)= Output tuples are 8 bytes long, bfr = 1024/8 = 128 B(U) = 10000/128 = 79 B(R) = / (1024/108) = 1112 Savngs = (B(R) - B(U))/B(R)*100% = 93% Estmatng Selecton Szes A selecton operator generally decreases the number of tuples n the output compared to the nput. By how much does the operator decrease the nput sze? The selectvty (sf) s the fracton of tuples selected by a selecton operator. Common cases and ther selectvtes:. 1) Equalty: S = (R) - sf= \/V(R,a) T(S) = T(R)/V(R,a).Reason: Based on the assumpton that values occur equally lkely n the database. However, estmate s stll the best on average even f the values v for attrbute a are not equally dstrbuted n the database.. 2) nequalty: S = (R) - sf= 1/3 T(S) = T(R)/3.Reason: On average, you would thnk that the value should be T(R)/2. However, queres wth nequaltes tend to return less than half the tuples, so the rule compensates for ths fact.. 3) Not equals: S = (R) - sf= 1 T(S) = T(R).Reason: Assume almost all tuples satsfy the condton Estmatng Jon Szes We wll study only estmatng the sze of natural jon. Other types of jons are equvalent or can be translated nto a cross-product followed by a selecton. The two relatons joned are R(X,Y) and S(Y,Z). We wll assume Y conssts of only one attrbute. The challenge s we do not know how the set of values of Y n R relate to the values of Y n S. There are some possbltes:. 1) The two sets are dsjont. Result sze = 0.. 2) Y may be a foregn key of R jonng to a prmary key of S. Result sze n ths case s T(R).. 3) Almost all tuples of R and S have the same value for F, so result sze n the worst case s T(R)*T(S).

10 13 Estmatng Operaton Cost Estmatng Jon Szes Example Example:. R(a,b) wth T(R) = 1000 and V(R,b) = 20.. S(b,c) wth T(S) = 2000, V(S,b) = 50, and V(S,c) = 100. U(c,d) wth T(U) = 5000 and V(U,c) = 500 Calculate the natural jon RP SMU. 1 )(RMsmu- T(RMS) = T(R) T(S)fmax{V(R, b), V(S, b)) = 1000 * 2000 / 50 = Now jon wth U. Fnal sze = T(R^S)*T(U)lmax[V(R^S,c),V(Utc))» * 5000 / 500 = 400, Estmatng Szes of other Operators The sze of the result of set operators, duplcate elmnaton, and groupng s hard to determne. Some estmates are below:. Unon.bag unon = sum of two argument szes.set unon = mnmum s the sze of the largest relaton, maxmum s the sum of the two relatons szes. Estmate by takng average of mn/max.. ntersecton.mnmum s 0, maxmum s sze of smallest relaton. Take average.. Dfference.Range s between T(R) and T(R) - T(S) tuples. Estmate: T(R) - \/2*T(S) Duplcate Elmnaton.Range s 1 to T(R). Estmate by ether takng smaller of 1/2*T(R) or product of all V(R,a) for all attrbutes a. Groupng.Range and estmate s smlar to duplcate elmnaton. 2.6 Cost-Based Query Optmzaton Cost-based optmzaton s used to determne the best physcal query plan gven a logcal query plan. The cost of a query plan n terms of dsk /Os s affected by:. 1) The logcal operatons chosen to mplement the query (the logcal query plan).. 2) The szes of the ntermedate results of operatons..3) The physcal operators selected.. 4) The orderng of smlar operatons such as jons.. 5) The method of passng arguments from one operator to another (ppelnng versus materalzaton) Obtanng Sze Estmates The cost calculatons for the physcal operators reled on reasonable estmates for B(R), T(R), and V(R,a). Most DBMSs allow an admnstrator to explctly request these statstcs be gathered. t s easy to gather them by performng a scan of the relaton. t s also common for the DBMS to gather these statstcs ndependently durng ts operaton. Note that by answerng one query usng a table scan, t can smultaneously update ts

11 estmates about that table! t s also possble to produce a hstogram of values for use wth V(R,a) as not all values are equally lkely n practce. Hstograms dsplay the frequency that attrbute values occur. Snce statstcs tend not to change dramatcally, statstcs are computed only perodcally nstead of after every update Usng Sze Estmates n Cost-Based Optmzaton Gven a logcal query plan, the smplest algorthm to determne the best physcal plan s an exhaustve search. n an exhaustve search, we evaluate the cost of every physcal plan that can be derved from the logcal plan and pck the one wth mnmum cost. The tme to perform an exhaustve search s extremely long because there are many combnatons of physcal operator algorthms, operator orderngs, and jon orderngs. Usng Sze Estmates n Cost-based Optmzaton (2) Snce exhaustve search s costly, other approaches have been proposed based on ether a top-down or bottom-up approach. Top-down algorthms start at the root of the logcal query tree and pck the best mplementaton for each node startng at the root. Bottom-up algorthms determne the best method for each subexpresson n the tree (startng at the leaves) untl the best method for the root s determned Optmzaton Choosng a Selecton Method n buldng the physcal query plan, we wll have to pck an algorthm to evaluate each selecton operator. Some of our choces are: table scan ndex scan There also may be several varants of each choce f there are multple ndexes. We evaluate the cost of each choce and select the best one Optmzaton Choosng a Jon Method n buldng the physcal query plan, we wll have to pck an algorthm to evaluate each jon operator:. nested-block jon - one-pass jon or nested-block jon used f reasonably sure that relatons wll ft n memory.. sort-jon s good when arguments are sorted on the jon attrbute or there are two or more jons on the same attrbute.. ndex-jon may be used when an ndex s avalable.. hash-jon s generally used f a multpass jon s requred, and no sortng or ndexng can be exploted Optmzaton Choosng other Operators Determnng the algorthms to select for the other operators s smlar. Ths ncludes the set operators. Projecton s always mplemented as a table scan, so no decsons must be made for that operator. 2.7 Concluson A query processor must frst transform a query nto nternal form, valdate ts syntax, then translate the query nto relatonal algebra (logcal query plan). Gven a relatonal algebra 14

12 15 query, there are many dfferent ways of evaluatng the query by usng ndces, sortng, and varous algorthms. Dsk access s the bottleneck that s often optmzed. Database statstcs allow query costs to be calculated Major Objectves The "One Thngs": Convert an SQL query to a parse tree usng a grammar. Convert parse tree to ogcal query tree for regular and nested queres. Use optmzaton (6 rules) and relatonal algebra laws to optmze logcal query trees. Convert logcal query tree to physcal query tree. Calculate estmates for estmatng operaton costs/szes for selecton, projecton, jons, and set operatons. Major Theme: The query optmzer uses heurstc (relatonal algebra laws) and cost-based optmzaton to greatly mprove the performance of query executon Objectves. Convert an SQL query to a parse tree usng a grammar.. Explan the dfference between syntax and semantc valdaton and the query prcessor component responsble for each.. Defne: vald parse tree, logcal query tree, physcal query tree. Convert parse tree to logcal query tree for regular and nested queres. Explan the dfference between correlated and uncorrelated nested queres. Use heurstc optmzaton (6 rules) and relatonal algebra laws to optmze logcal query trees,.selecton laws (splttng law), projecton laws, laws for jons, duplcate elmnaton, and groupng, equvalence preservng transformatons