Magda Balazinska - CSE 444, Spring

Transcription

1 Query Optimization Algorithm CE 444: Database Internals Lectures Query Optimization (part 2) Enumerate alternative plans (logical & physical) Compute estimated cost of each plan Compute number of I/Os Compute CPU cost Choose plan with lowest cost his is called cost-based optimization 1 2 Lessons Need to consider several physical plans Even for one, simple logical plan No magic best plan: depends on the data earch space Outline Algorithm for enumerating query plans In order to make the right choice Need to have statistics over the data he B s, the s, the V s 3 4 elational Algebra Equivalences elections Commutative: σ c1 (σ c2 ()) same as σ c2 (σ c1 ()) Cascading: σ c1 c2 () same as σ c2 (σ c1 ()) Projections Cascading Joins Commutative : same as Associative: ( ) same as ( ) Left-deep plan Left-Deep Plans and Bushy Plans Bushy plan 6 1

2 Commutativity, Associativity, Distributivity Laws Involving election =, ( ) = ( ) =, ( ) = ( ) σ C AND C () = σ C (σ C ()) = σ C () σ C () σ C O C () = σ C () σ C () σ C ( ) = σ C () ( ) = ( ) ( ) σ C ( ) = σ C () σ C ( ) = σ C () σ C () σ C ( ) = σ C () Assuming C on attributes of 7 8 Example: imple Algebraic Laws Laws Involving Projections Example: (A, B, C, D), (E, F, G) σ F=3 ( D=E ) =? σ A=5 AND G=9 ( D=E ) =? Π M ( ) = Π M (Π P () Π Q ()) Π M (Π N ()) = Π M () /* note that M N */ Example (A,B,C,D), (E, F, G) Π A,B,G ( D=E ) = Π? (Π? () D=E Π? ()) 9 10 Laws involving grouping and aggregation δ(γ A, agg(b) ()) = γ A, agg(b) () γ A, agg(b) (δ()) = γ A, agg(b) () if agg is duplicate insensitive Which of the following are duplicate insensitive? sum, count, avg, min, max Laws Involving Constraints Product(pid, pname, price, cid) Company(cid, cname, city, state) Foreign key Π pid, price (Product cid=cid Company) = Π pid, price (Product) γ A, agg(d) ((A,B) B=C (C,D)) = γ A, agg(d) ((A,B) B=C (γ C, agg(d) (C,D)))

3 earch pace Challenges earch space is huge! Many possible equivalent trees Many implementations for each operator Many access paths for each relation File scan or index + matching selection condition earch space Outline Algorithm for enumerating query plans Cannot consider ALL plans Heuristics: only partial plans with low cost Key Decisions Logical plan What logical plans do we consider (left-deep, bushy?); earch pace Which algebraic laws do we apply, and in which context(s)?; Optimization rules In what order do we explore the search space?; Optimization algorithm Key Decisions Physical plan What physical operators to use? What access paths to use (file scan or index)? Pipeline or materialize intermediate results? hese decisions also affect the search space wo ypes of Optimizers hree Approaches to earch pace Enumeration Heuristic-based optimizers: Apply greedily rules that always improve plan ypically: push selections down Very limited: no longer used today Complete plans Bottom-up plans Cost-based optimizers: Use a cost model to estimate the cost of each plan elect the cheapest plan We focus on cost-based optimizers op-down plans

4 Complete Plans Bottom-up Partial Plans (A,B) (B,C) (C,D) ELEC * FOM,, WHEE.B=.B and.c=.c and.a<40 19 Why is this search space inefficient? (A,B) (B,C) (C,D) Why is this better? ELEC * FOM,, WHEE.B=.B and.c=.c and.a< (A,B) (B,C) (C,D) ELEC * FOM, WHEE.B=.B and.a < 40 op-down Partial Plans ELEC * FOM,, WHEE.B=.B and.c=.c and.a<40 ELEC * FOM WHEE.A < 40 ELEC.A,.D FOM,, WHEE.B=.B and.c=.c.. wo ypes of Plan Enumeration Algorithms Dynamic programming (in class) Based on ystem (aka elinger) style optimizer[1979] Limited to joins: join reordering algorithm Bottom-up ule-based algorithm (will not discuss) Database of rules (=algebraic laws) Usually: dynamic programming Usually: top-down Originally proposed in ystem [1979] Only handles single block queries: ELEC list ome heuristics for search space enumeration: elections down Projections up Avoid cartesian products earch space = join trees Algebraic laws = commutativity, associativity Algorithm = dynamic programming J

5 elinger Optimizer Algorithm Original elinger optimizer enumerates different logical and physical plans at the same time 1 2. n Join tree: Join rees o simplify the discussion, we will first study the approach considering only logical plans We come back to the actual elinger enumeration algorithm at the end of the lecture A plan = a join tree A partial plan = a subtree of a join tree ypes of Join rees ypes of Join rees Bushy: ight deep: ypes of Join rees Left deep: Work well with existing join algos Nested-loop and hash-join Facilitate pipelining Join ordering: Given: a query n Find optimal order 3 1 elinger algorithm considers only those trees Dynamic programming can be used with all trees Assume we have a function cost() that gives us the cost of every join tree ELEC list

6 For each subquery Q {1,, n} compute the following: ize(q) = the estimated size of Q I.e., the cardinality of the result of Q Plan(Q) = a best plan for Q Cost(Q) = the estimated cost of that plan Note: we focus first on logical plans so we will use as cost estimate the sum of cardinalities of intermediate relations ELEC list tep 1: For each { i }, set: ize({ i }) = ( i ) Plan({ i }) = i hat s the only alternative for a logical plan Cost({ i }) = 0 emember that we are computing costs of logical plans ELEC list tep 2: For each Q { 1,, n } involving i relations: ize(q) = estimate it recursively For every pair of subqueries Q, Q s.t. Q = Q Q compute cost(plan(q ) Plan(Q )) Cost(Q) = the smallest such cost Plan(Q) = the least-cost plan tep 3: eturn Plan({ 1,, n }) ELEC list ELEC list Example Example We use cost model for logical plans: Cost(P 1 P 2 ) = Cost(P 1 ) + Cost(P 2 ) + size(intermediate results for P 1, P 2 ) Cost of a scan = 0 U Assumptions: () = 2000 () = 5000 () = 3000 (U) = 1000 ELEC * FOM,,, U WHEE cond 1 AND cond 2 AND... All join selectivities = 1% ( ) = 0.01*()*() ( ) = 0.01*()*() etc

7 ubquery ize Cost Plan ubquery ize Cost Plan () = 2000 () = 5000 () = 3000 (U) = 1000 U () = 2K () = 5K () = 3K (U) = 1K 100k 0 60k 0 U 20k 0 U 150k 0 U U 50k 0 U U U 30k 0 U 3M 60k () U U 1M 20k (U) U U 600K 20k (U) U U 1.5M 30k (U) U 37 U 30M 60k +50k=110k ()(U) 38 educing the earch pace : ummary estriction 1: only left linear trees (no bushy) estriction 2: no trees with cartesian product (A,B) (B,C) (C,D) Why? Plan: ((A,B)(C,D)) (B,C) has a cartesian product. Most query optimizers will not consider it Handles only join queries: elections are pushed down (i.e. early) Projections are pulled up (i.e. late) akes exponential time in general, BU: Left linear joins may reduce time Non-cartesian products may reduce time further Completing the Physical Query Plan Choose algorithm for each operator How much memory do we have? Are the input operand(s) sorted? Access path selection for base tables Decide for each intermediate result: o materialize o pipeline More about the elinger Algorithm elinger enumeration algorithm considers Different logical and physical plans at the same time Cost of a plan is IO + CPU Concept of interesting order during plan enumeration ame order as that requested by ODE BY or GOUP GY Attributes that appear in equi-join predicates hey can speed-up a sort-merge join later

8 More about the elinger Algorithm tep 1: Enumerate all access paths for a single relation File scan or index scan Keep the cheapest for each interesting order elinger Algorithm Example On the white board tep 2: Consider all ways to join two relations Use result from step 1 as the outer relation Consider every other possible relation as inner relation Estimate cost when using sort-merge or nested-loop join Keep the cheapest for each interesting order teps 3 and later: epeat for three relations, etc