Extensible Query Processing in Starburst

Size: px

Start display at page:

Download "Extensible Query Processing in Starburst"

Asher Dalton
6 years ago
Views:

1 Extensible Query Prcessing in Starburst Laura M. Haas, J.C. Freytag, G.M. Lhman, and H.Pirahesh IBM Almaden Research Center CS848 Instructr: David Tman Presented By Yunpeng James Liu

2 Outline Intrductin Language Overview f Language Prcessing Language Prcessing -Query Graph Mdel -Query Rewrite -Plan Optimizatin Query Evaluatin System Cnclusins

3 Intrductin Tday s DBMSs are unable t supprt the increasing demands f the varius applicatins that wuld like t use a DBMS. Engineering, ffice and gegraphic applicatins have attempted t use relatinal databases t stre their data. Hwever, the demands f each kind f applicatins fr new data types and functins are different. Traditinal DBMS culd nt satisfy these demands. Therefre, sme researchers designed Starburst.

4 Intrductin What is Starburst? A: Starburst is a prject at IBM Almaden Research Center aiming t extend relatinal DBMS technlgy t bridge the gap between applicatins and the DBMS. Gal f Starburst: Extensibility, r the ability t tailr the DBMS cde t supprt particular applicatins r hardware. DBC: Extensins will be made by knwledgeable database implementers, we call them DBC(Database Custmizers).

5 Intrductin Starburst cntains: --Query Language Prcessr: Crna, cmpile the user s query int an efficiently interpretable frm. --Data Manager: Cre, manage and maintain data, and Crna invkes the data manager s services as needed.

6 Language Starburst s language, Hydrgen, is based n SQL. Hydrgen includes mst SQL features, such as: aggregatin, set peratins and nested subqueries. Language Extensins: It als Expands n SQL, such as: it adds new functinality t the language (next slide).

Defined functins: Language --Scalar functins (e.g., Area(Width, Length)) In: ne r mre field values frm a single tuple Out: a single value --Aggregate functins (e.g., StandardDeviatin(Salary)) In: ne r mre field values frm several tuples Out: a single value --Set predicate functins (e.

7 Defined functins: Language --Scalar functins (e.g., Area(Width, Length)) In: ne r mre field values frm a single tuple Out: a single value --Aggregate functins (e.g., StandardDeviatin(Salary)) In: ne r mre field values frm several tuples Out: a single value --Set predicate functins (e.g., T1.c1=T2.c1 and SELECT T2.c1 FROM T2 WHERE T2.c2=5) In: a simple predicate and a subquery (defines the range fr the predicate) Out: a blean value --Table functins (e.g., Sample(table, int)) In: ne r several table expressins as well as field values Out: a relatin (new table)

8 Overview f Language Prcessing Tw stages: cmpilatin f the query and executin

9 Language Prcessing The Query Graph Mdel: DBC can use as much f Crna s functins as he can; DBC als can write the rest functins himself by defining a generic internal representatin fr queries and giving the DBC access t this representatin. We call this representatin QGM. Let us use the fllwing query and its QGM (next slide) t illustrate the mst imprtant QGM cnstructs: SELECT partn, price, rder_qty FROM qutatins Q1 WHERE Q1.partn IN (SELECT partn FROM inventry Q3 WHERE Q3.nhand_qty < Q1.rder_qty AND Q3.type = cpu )

10 Language Prcessing Bxes (select peratins) Head: utput table Bdy: represents the peratin Select Operatins Outer: qutatins table + a predicate ver result f a subquery. Inner: inventry + a predicate with tw cnjuncts. Edges Range edges: each select accesses t either a stred r derived table, which is represented by a vertex and a dtted line. Qualifier edges: each cnjunct f a predicate is represented by a rectangle n a slid line cnnecting ne r mre vertexes.

11 Language Prcessing Therefre, Vertices, edges (range and qualifier) and bxes (with heads) are main cnstructs f QGM. They are extremely pwerful, allwing us t express arbitrarily cmplicated queries. Next phase f query prcessing is query rewrite. Why relatinal DBMSs need t rewrite queries? --A: Because query rewrite is essentially a frm f ptimizatin. Fr example, views may be merged with the queries that use them; redundant jins may be eliminated; predicates may be migrated frm ne peratin t anther t minimize the amunt f data accessed.

12 Language Prcessing Query Rewrite Starburst has its wn rule system fr query rewrite, which cnsists f tw parts, the cnditin and the actin. Starburst prvides a set f rules fr rewriting the base system peratins. A DBC may prvide additinal rules fr these peratins. Rules may be gruped int rule classes t limit the number f rules that have t be examined, t allw mdularizatin f rules, and t give the DBC mre explicit cntrl ver the executin sequence.

13 Language Prcessing Example f Query Rewrite

14 Language Prcessing Query Rewrite Besides the rewrite rules, Starburst s rule system has tw ther majr cmpnents: the rule engine and the search facility. Rule engine is designed t meet the needs f query rewrite rules in general. Several cntrl strategies are prvided: sequential, pririty and statistical. Search strategy is independent f bth rules and the rule engine, whse rle is t brwse thrugh QGM, prviding the cntext fr the rules t wrk n. Query rewrite must interact with plan ptimizatin t estimate the cst f alternative QGM representatins.

15 Language Prcessing Plan Optimizatin Given a QGM representatin f the query, the ptimizer estimates the cst f alternative query evaluatin plans (QEPs) t execute that query, and chses the cheapest. Three majr aspects f ptimizatin: 1. Plan generatin; 2. Plan csting, and 3. search strategy. Executable plans are defined using a grammar-like set f parameterized prductin rules called strategy alternative rules (STARs). These rules define higher-level nn-terminal cnstructs frm lw-level terminal (LOLEPOP, lw level plan peratr) database peratrs. Terminals (lw-level plan peratrs) Nn-terminals (Star) In summary, a Starburst query evaluatin plan (QEP) is a nesting f invcatins f LOLEPOPs.

16 Language Prcessing Plan Optimizatin The initial Star-based plan generatr cntains: -- a general-purpse STAR evaluatr; -- a search strategy that chses the next STAR t evaluate; -- An array f STARS. This design permits the ptimizer designer t add, change r delete rules in the STAR array withut affecting the cde fr the search engine r the rule evaluatr. Query Evaluatin System: the last phase f query prcessing in Starburst.

17 Query Evaluatin System At runtime, Starburst s query evaluatin system (QES) takes a query evaluatin plan (QEP), and evaluates it against the database. This is the whle prcess f query prcessing in Starburst.

18 Cnclusin This paper described the verall design f the Starburst language prcessing cmpnent. The tw cntributins f Starburst: Revisited internal data structures --Query graph mdel --Query executin plan: lw-level peratrs and STARs Mechanisms fr extensibility --Rules fr query rewrite and plan ptimizatin

19 Thanks fr listening! Any Questins?

Querying Data with Transact SQL

Querying Data with Transact SQL Curse Cde: 20761 Certificatin Exam: 70-761 Duratin: 5 Days Certificatin Track: MCSA: SQL 2016 Database Develpment Frmat: Classrm Level: 200 Abut this curse: This curse is