The Volcano Optimizer Generator: Extensibility and Efficient Search

Transcription

1 The Volcano Optimizer Generator: Extensibility and Efficient Search - Prithvi Lakshminarayanan

2 Athors Goetz Graefe, Portland State University Won the Most Inflential Paper award in 1993 Worked at HP, Microsoft and crrently at Google William J. McKenna, University of Colorado at Bolder

3 Flashback to 1993

4 Developments in 1993 ODBMS

5 Oracle`s annal report

6 Problem

7 Problem Increase in data volmes Acceptance problems in emerging database applications Optimization- Speed in which the data is being retrieved Parallelization- Improving performance Improved optimizer generator over EXODUS

8 EXODUS Compiles an optimizer based on a set of rles Every expressions are stored in a MESH Pros: Cons: Very extensible architectre Logical and Physical operators in the same MESH Cost ineffective Ref paper: The EXODUS optimizer generator, G.Graefe, 1993

9 Logical and Physical operators Logical Operators: Logical operators test for the trth of some condition Retrns a Boolean data type with a vale of TRUE, FALSE, or UNKNOWN Example- ALL, AND, ANY Physical Operators: Physical operators implement the operation described by logical operators. Each physical operators perform an operation Example- INIT(), GETNEXT(),CLOSE()

10 It has to Act as a stand alone Be more efficient in time and memory consmption Provide effective, efficient and extensible spport Permit the se of data models and heristics Flexible cost models VOLCANO

11 What is Qery Optimization? Optimizer generators take in a model of qeries and plans and otpt a qery optimizer that performs the actal optimization. Qery optimizer attempts to determine the most efficient way to execte a given qery It prodces an exection plan and code generator to execte

12 The Generator Paradigm Optimizer Implementer Person who specifies the data model and implements the DBMS software DBMS User- Person poses qeries to be optimized and exected

13 Volcano optimizer at a glance Database implementers Framework Physical and Logical Models Volcano optimizer generator Optimized Plan

14 Design Principles Relational algebra being sed to spport Object Oriented Systems Logical algebra Qery Physical algebra- Qery evalation plan which has the algorithms Creating Rles to ensre modlarity knowledge abot patterns in a concise and modlar fashion knowledge of algebraic laws as reqired for eqivalence transformations Inpt as algebraic eqivalences Rle compilation over interpretation de to faster exection Dynamic programming

15 Optimizer Generator Inpt and Optimizer Operation User qeries as algebraic expressions Optimization is mapping a logical algebra expression into the optimal eqivalent physical algebra expression Implementation algorithm is chosen Transformation and implementation rles for complex mappings Generates a logical expression and a physical property vector Cost fnction to measre CPU time, total elapsed time

16 Steps in Optimizer Generator Inpt and Optimizer Operation User qeries The optimizer reorders operators and selects implementation algorithms Check if the rles get satisfied Cost fnction gets evoked Optimizer generator gets bilt

17 Search Algorithm Transformed sing a Transformation rle Algorithms can deliver Logical expressions with Physical properties Enforcer might be sefl to permit additional algorithm choices

18 The Search Engine Search engine can be sed in all optimizers Dynamic programming and very goal oriented than EXODUS Backward chaining - explores only the sbqeries and plans that trly participate in a larger expression A Hash table is sed to prevent redndant optimization

19 Associativity Rle A and B are eqivalent Expression C is not eqivalent to any expression on the left Hence, a new eqivalence is created and optimized This is done for the cost effectiveness and optimization in of expression B

20 Comparing with EXODUS The EXODUS code was messy and slow EXODUS did not distingish logical from physical algebra which led to some inefficiencies EXODUS had no notion of physical properties EXODUS did not have a generic cost fnction EXODUS did not have the ability to modify search strategy like volcano Volcano took less time to optimize

21 EXODUS Vs Volcano EXODUS Works on a MESH which had logical and physical expressions in them Physical and logical properties worked bad together Bottom-p approach Cost is not well defined Cannot be made extensible Volcano Clearly able to distingish logical and physical expressions Physical and logical properties worked good together Top-down approach; sb expressions are optimized only if wanted Cost is well defined for optimizer implementation More extensible

22 Average optimization effort Solid line Sn SparcStation-1 with 12 MIPS Dashed line indicates the estimated plan exection time X-axis is the 8 binary joins and Y- axis is the logarithmic

23 Reslts and findings The Volcano-generated optimizer performed exhastive search for all qeries with less than 1 MB of work space The increase of Volcano's optimization costs is abot exponential The Volcano optimizer generator is not only more extensible, it is also mch more efficient and effective than the earlier EXODUS

24 Spark SQL`s Catalyst Spark SQL is one of the newest and most technically involved components Catalyst optimizer leverages advanced programming langage featres Helpfl in advanced analytics and for external developers It offers perform analysis, optimization, planning, and rntime code generation.

25 Spark SQL`s Catalyst

26 Smmary Satisfies higher fnctionality and performance by having efficient tools for qery processing Qery processing engine independent of any data model Algebraic eqivalence rles are sitable for qery optimization Volcano optimizers are better than EXODUS

27 Qestions from me! Was the performance and fnctionality tradeoff satisfied in Volcano? The paper does not speak abot the drawbacks of volcano when compared with EXODUS What are the cases when the optimization fails in Volcano? What next after volcano?

28 Qestions for me?