Axiomatic Foundations and Algorithms for Deciding Semantic Equivalences of SQL Queries

Transcription

1 Axiomaic Foundaions and Algorihms for Deciding Semanic Equivalences of SQL Queries Shumo Chu, Brendan Murphy, Jared Roesch, Alvin Cheung, Dan Suciu Paul G. Allen School of Compuer Science and Engineering Universiy of Washingon {chushumo, jroesch, akcheung, ABSTRACT Deciding he equivalence of SQL queries is a fundamenal problem in daa managemen. As prior work has mainly focused on sudying he heoreical limiaions of he problem, very few implemenaions for checking such equivalences exis. In his paper, we presen a new formalism and implemenaion for reasoning abou he equivalences of SQL queries. Our formalism, U- semiring, exends SQL s semiring semanics wih unbounded summaion and duplicae eliminaion. U-semiring is defined using only very few axioms and can hus be easily implemened using proof assisans such as Lean for auomaed query reasoning. Ye, hey are sufficien enough o enable us reason abou sophisicaed SQL queries ha are evaluaed over bags and ses, along wih various inegriy consrains. To evaluae he effeciveness of U-semiring, we have used i o formally verify 68 equivalen queries and rewrie rules from boh classical daa managemen research papers and real-world SQL engines, where many of hem have never been proven correc before. PVLDB Reference Forma: Shumo Chu, Brendan Murphy, Jared Roesch, Alvin Cheung, Dan Suciu. Axiomaic Foundaions and Algorihms for Deciding Semanic Equivalences of SQL Queries. PVLDB, 11 (11): , DOI: hps://doi.org/ / INTRODUCTION All modern relaional daabase managemen sysems (DBMSs) conain a query opimizer ha chooses he bes means o execue an inpu query. A he hear of an opimizer is a rule-based rewrie sysem ha uses rewrie rules o ransform he inpu SQL query ino anoher (hopefully more efficien) query o execue. A key challenge in query opimizaion is how o ensure ha he rewrien query is indeed semanically equivalen o he inpu, i.e., ha he original and rewrien queries reurn he same resuls when execued on all possible inpu daabase insances. Hisory suggess ha he lack of ools o esablish such equivalences has caused longsanding, laen bugs in major daabase sysems [32], and new bugs coninue o arise [7, 10]. All such errors lead o incorrec resuls reurned ha can cause dire consequences. Permission o make digial or hard copies of all or par of his work for personal or classroom use is graned wihou fee provided ha copies are no made or disribued for profi or commercial advanage and ha copies bear his noice and he full ciaion on he firs page. To copy oherwise, o republish, o pos on servers or o redisribue o liss, requires prior specific permission and/or a fee. Aricles from his volume were invied o presen heir resuls a The 44h Inernaional Conference on Very Large Daa Bases, Augus 2018, Rio de Janeiro, Brazil. Proceedings of he VLDB Endowmen, Vol. 11, No. 11 Copyrigh 2018 VLDB Endowmen /18/07. DOI: hps://doi.org/ / In he pas, query opimizers were developed by only small number of commercial daabase vendors wih dedicaed eams o handle cusomer repored bugs. Such eams effecively aced as manual solvers for query equivalences. The explosive growh of new daa analyic sysems in recen decades (e.g., Spark SQL [12], Hive [4], Dremel [42], Myria [39, 55], among many ohers) has unforunaely exacerbaed his problem significanly, as many such new sysems are developed by eams ha simply lack he dedicaed resources o check every single rewrie implemened in heir sysem. For example, he Calcie open source query processing engine [1] is mainly developed by open-source conribuors. While i conains more han 232 rewrie rules in is opimizaion engine, none of which has been formally validaed. The same holds rue for similar engines. On he oher hand, deciding wheher wo arbirary SQL queries are semanically equivalen is a well-sudied problem in he daa managemen research communiy ha has shown o be undecidable in general [11]. Mos subsequen research has been direced a idenifying fragmens of SQL where equivalence is decidable, under se semanics [17, 47] or bag semanics [24]. This line of work, focused on heoreical aspecs of he problem [51, 47, 41], led o very few implemenaions, mos of which were resriced o applying he chase procedure o conjuncive queries [14]. An alernaive approach o query equivalence was recenly proposed by he COSETTE sysem [23] based on a new SQL semanics. I inerpres SQL relaions as K-relaions [35]. A K-relaion is semiring ha maps each uple o a value R() ha denoes he mulipliciy of he uple in he relaion. Normally, he mulipliciy is an ineger, hus he semiring K is he semiring of naural numbers N, bu in order o suppor some of he SQL consrucs (such as projecions, wih or wihou DISTINCT), COSETTE resors o using a much more complex semiring, namely ha of all univalen ypes in Homoopy Type Theory (HoTT) [50]. Equivalence of wo SQL queries is hen proven in COSETTE using he Coq proof assisan exended wih he HoTT library [36]. While he sysem can handle many feaures of SQL, is reliance on HoTT makes i difficul o exend as HoTT is boh a Coq library under developmen and an acive research area iself. As a resul, COSETTE has a number of shorcomings. For example, COSETTE does no suppor foreign keys as hey are difficul o model using he HoTT library in Coq. In his paper, we propose a new algebraic srucure, called he unbounded semiring, or U-semiring. We define he U-semiring by exending he sandard commuaive semiring wih a few simple consrucs and axioms. This new algebraic srucure serves as a foundaion for our new formalism ha models he semanics of SQL. To prove he semanic equivalence of wo SQL queries, we firs conver hem ino U-semiring expressions, i.e., U-expressions. Deciding he equivalence of queries hen becomes deermining he 1482

2 equivalence of wo U-expressions which, as we will show, is much easier compared o classical approaches. Our core conribuion is idenifying he minimal se of axioms for U-semirings ha are sufficien o prove sophisicaed SQL query equivalences. This is imporan as he number of axioms deermines he size of he rused code base in any proof sysem. As we will see, he few axioms we designed for U-semirings are surprisingly simple as hey are jus ideniies beween wo U-expressions. Ye, hey are sufficien o prove various advanced query opimizaion rules ha arise in real-world opimizers. Furhermore, we show how inegriy consrains (ICs) such as keys and foreign keys can be expressed as U-expressions ideniies as well, and his leads o a single framework ha can model many differen feaures of SQL and he relaional daa model. This allows us o devise differen algorihms for deciding he equivalences of various ypes of SQL queries including hose ha involve views, or leverage ICs o rewrie he inpu SQL query as par of he proof. To ha end, we have developed a new algorihm, UDP, o auomaically decide he equivalence of wo arbirary SQL queries ha are evaluaed under mixed se/bag semanics, 1 and also in he presence of indexes, views, and oher ICs. A a high level, our algorihm performs rewries reminiscen o he chase/back-chase procedure [45] bu uses U-expressions raher han firs order logic senences. I performs a number of ess for isomorphisms or homomorphisms o capure he mixed se/bag semanics of SQL. Our algorihm is sound in general and is complee in wo resriced cases: when he wo queries are Unions of Conjuncive Queries (UCQ) under se semanics, or UCQ under bag semanics. We implemen UDP on op of he Lean proof assisan [26]. The implemenaion includes he modeling of relaions and queries as U-expressions, axiomaic represenaions of ICs as simple U-expression ideniies, and he algorihm for checking he equivalence of U-expressions. We evaluae UDP using various opimizaion rules from classical daa managemen research lieraure and as implemened in he Apache Calcie framework [1]. These rules consis of sophisicaed SQL queries wih a wide range of feaures such as subqueries, grouping and aggregae, DISTINCT, and inegriy consrains.in fac, only 1 of hem have been proven before. UDP can formally and auomaically prove mos of hem (39 of 45). The running ime of UDP on each of hese 39 rules is wihin 30 seconds. Our sysem only proves equivalence. In prior work [22] we described he complemenary ask ha uses a model checker o find counerexamples o idenify buggy rewries [32, 7, 10]; of course, he model checker canno prove equivalence, which is our curren focus. In summary, our paper makes he following conribuions: We describe a new algebraic srucure, he U-semiring, which exends he sandard semiring wih he necessary operaors o model he semanics of a wide range of SQL queries (Sec. 3). We propose a new formalism for expressing differen kinds of inegriy consrains over a U-semiring. We implemen such consrains using a number of axioms (in he form of simple ideniies) such ha hey can be easily implemened using a proof assisan. Doing so allows us o easily uilize hem in equivalence proofs (Sec. 4). We describe a new algorihm for deciding he equivalences of differen ypes of SQL queries. The algorihm operaes enirely on our represenaion of SQL queries as U-expressions. We show 1 Mixed se/bag semanics is he bag semanics ha allows explici DISTINCT on arbirary subqueries, bag semanics and se semanics are special cases of mixed se/bag semanics. Equivalen SQL Queries SELECT * FROM R WHERE.a >= Q1 SELECT 2.* FROM I 1, R 2 -- Q2 WHERE 1.k = 2.k AND 1.a >= 12 where k is a key of R, and I is an index on R defined as: I := SELECT 3.k AS k, 3.a AS a FROM R 3 Corresponding Equivalence in Semirings Q 1 () = λ. R () [.a 12] Q 2 () = λ. [ 2 = ] [ 1.k = 2.k] [ 1.a 12] 1, 2, 3 [ 3.k = 1.k] [ 3.a = 1.a] R ( 3 ) R ( 2 ) Figure 1: Proving ha a query is equivalen o a rewrie using an index I requires proving a suble ideniy in a semiring. ha his algorihm is sound for general SQL queries and is complee for Unions of Conjuncive Queries under se and bag semanics (Sec. 5). We have implemened UDP using he Lean proof assisan, and have evaluaed UDP by collecing 69 real-world rewrie rules as benchmarks, boh from prior research work done in he daabase research communiy, and from Apache Calcie [1], an open-source relaional query opimizer. To our knowledge, he majoriy of hese rules have been never proven correc before, while UDP can auomaically prove 62 of 68 correc ones and fail as inended on he 1 buggy one (Sec. 6). 2. OVERVIEW We moivae our new semanics using an example query rewrie. As shown in Fig. 1, he rewrie changes he original query Q1 by using an index I for look up. We follow he GMAP framework [52], where an index is considered as a view definiion, and a query plan ha uses he index I o access he relaion R is represened logically by a query ha selecs from I hen joins on he key of R. Our goal is o devise a semanics for SQL along wih various inegriy consrains ha can be easily implemened as a ool for checking query equivalences. Unforunaely, he SQL sandard [25] is expressed in English and is difficul o implemen programmaically. One recen aemp is Q*cer [13], which models SQL using NRA (Nesed Relaional Algebra). NRA is implemened using he Coq proof assisan, wih relaions modeled using liss. To prove ha Q 1 and Q 2 are equivalen in Q*cer, users need o prove ha he oupu from he wo are equal up o elemen reordering and duplicae eliminaion (for se semanics). As Q*cer models relaions as liss, his amouns o wriing an inducive proof on he size of R, i.e., if R is empy, hen Q 1 oupus he same relaion as Q 2; if R is of size n and he wo oupus are equivalen, hen he wo oupus are also equivalen if R is of size n + 1. Wriing such proofs can be edious. As an illusraion, Q*cer requires 45 lines of Coq o prove ha selecion can be disribued over union [9] while, as we will see, using U-semiring only requires 1 line of Lean: disribue muliplicaion over addiion, which is one of he semiring axioms. Anoher formalism proposed by Guagliardo e al. [38] models relaions as bags. While i models NULL semanics, i (like Q*cer) does no model inegriy consrains, which are widely used in almos all daabase sysems and are involved in he rewries in many real world opimizers. There is no known algorihm based on heir formalism o auomaically check he equivalence of SQL queries. We insead base our semanics on K-relaions, which was firs proposed by Green e al [35]. Under his semanics, a relaion is 1483

3 modeled as a funcion ha maps uples o a commuaive semiring, K = (K, 0, 1, +, ). In oher words, a K-relaion, R, is a funcion: R :Tuple(σ) K wih finie suppor. Here, Tuple(σ) denoes he (possibly infinie) se of uples of ype σ, and R () represens he mulipliciy of in relaion R. For example, a relaion under SQL s sandard bag semanics is an N-relaion (where N is he semiring of naural numbers), and a relaion under se semanics is a B-relaion (where B is he semiring of Booleans). All relaional operaors and SQL queries can be expressed in erms of semiring operaions; for example: SELECT * FROM R x, S y = λ ( 1, 2). R ( 1) S ( 2) SELECT * FROM R WHERE a > 10 = λ. [.a > 10] R () SELECT a FROM R = λ. [.a = ] R ( ) :Tuple(σ) For any predicae b, we denoe [b] he elemen of he semiring defined by [b] = 1 if he predicae holds, and [b] = 0 oherwise. K-relaions can be used o prove many simple query rewries by reducing query equivalences o semiring equivalences, which are much easier since one can use algebraic reasoning raher han proofs by inducion. However, for sophisicaed rewries like he one shown in Fig. 1, K-relaions are no sufficien o prove query equivalences, le alone auomae he proof search. One problem is ha K-relaion does no model inegriy consrains (keys, foreign keys, ec.), which are usually expressed as generalized dependencies [11] (i.e., logical formulas of he form x(ϕ yψ)). For example, consider he wo queries in Fig. 1: Q 1 is a selecion on R.a 12, while Q 2 rewries he given query using an index I. These wo queries are indeed semanically equivalen in ha scanning a able using a given aribue (a in he example) is he same as performing an index scan on he same aribue. However, consider heir corresponding expressions over K-relaions, shown a he boom of Fig. 1; i is unclear how o express formally he fac ha R.k is a key, ye alone prove auomaically ha hese wo expressions are equal. To exend K-relaions for auomaically proving SQL equivalences under daabase inegriy consrains, we develop a novel algebraic srucure, U-semiring, as he semiring in K-relaions. A U-semiring exends a semiring wih hree new operaors,,, no( ), and a minimal se of axioms, each of which is a simple ideniy 2 ha is easy o implemen using a proof assisan. We also discover a se of U-semiring ideniies ha models daabase inegriy consrains such as keys and foreign keys. Wih hese addiions, SQL equivalences can be reasoned by checking he equivalences of heir corresponding U-expressions using only hese axioms expressed in U-semiring ideniies. Using our semanics, he rewrie shown in Fig. 1 can be proved by rewriing he Q 2 ino he Q 1 using U-semiring axioms in hree seps. Firs, he sum over 1 can be removed by applying he axiom of he inerpreaion of equaliy over summaion (Eq. (15)): λ. [ 2 = ] [ 3.k = 2.k] [ 3.a 12] R( 3) R( 2) 2, 3 Since k is a key of R, applying he U-semiring definiion of he key consrain (Def. 4.1) we ge: [ 3.k = 2.k] R ( 3) R ( 2) = [ 3 = 2] R ( 3) 2 An axiom is a logical senence, such as xr(x) S(x). An ideniy, or an equaional law, is an equaliy, such as x + y = y + x. The implicaion problem for ideniies is much simpler han for arbirary axioms. Tradiional generalized dependencies [11] are expressed as axioms, x(ϕ yψ), and only apply o queries under se semanics. h Program ::= s 1 ;... ; s n; s Saemen ::= verify q 1 q 2 schema σ(a 1 : τ 1,..., a n : τ n) able r(σ) key r(a 1,..., a n); foreign key r 1 (a 1,..., a n ) references r 2 (a 1,..., a n); view v q; index i on σ(a 1,..., a n); a Aribue ::= sring q Query ::= r SELECT p q FROM q 1 x 1,..., q n x n q WHERE p q 1 UNION ALL q 2 DISTINCT q x TableAlias ::= sring b Predicae ::= e 1 = e 2 NOT b b 1 AND b 2 b 1 OR b 2 TRUE FALSE EXISTS q e Expression ::= x.a f(e 1,..., e n) agg(q) p Projecion ::= * x.* e AS a p 1, p 2 f UDF, agg UDA ::= sring Figure 2: SQL fragmen suppored by our semanics Thus, he Q 2 can be rewrien o: λ. [ 2 = ] [ 3.a 12] R ( 2) [ 2 = 3] 2, 3 Applying Eq. (15) again o he above makes Q 2 is equivalen o Q 1. We presen a deailed proof in Ex 4.7. We nex explain hese axioms in deail and show how we use hem o develop an algorihm o formally and auomaically check he equivalences of general SQL queries. 3. AXIOMATIC FOUNDATIONS In his secion we inroduce a new algebraic srucure, U-semiring, and show ha i can be used o give semanics o SQL queries. 3.1 U-semirings Under sandard bag semanics, a SQL query and is inpu relaions can be modeled as N-relaions [35], i.e., relaions over he semiring (N, 0, 1, +, ). However, as we saw SQL queries include consrucs ha are no direcly expressible in a semiring, such as projecion (requiring an unbounded summaion), DISTINCT, and non-monoone operaors (e.g., NOT EXISTS). We now define a new algebraic srucure ha can be used for such purposes. DEFINITION 3.1. An unbounded semiring, or U-semiring, is (U, 0, 1, +,,, no( ), ( D )D D), where: (U, 0, 1, +, ) forms a commuaive semiring. 3 is a unary operaion called squash ha saisfies: 0 = 0, 1 + x = 1 (1) x + y = x + y (2) x y = x y (3) x x = x (4) x x = x (5) x 2 = x x = x (6) 3 Recall ha he semiring axioms are: associaiviy and commuaiviy of + and, ideniy of 0 for +, ideniy of 1 for, disribuiviy of over +, and 0 x = 0; see [35] for deails. 1484

4 no( ) is a unary operaion ha saisfies he following: no(0) = 1 no(x y) = no(x) + no(y) no(x + y) = no(x) no(y) no( x ) = no(x) = no(x) D is a se of ses; each D D is called a summaion domain. For each D, he operaion D : (D U) U is called an unbounded summaion: is inpu is a funcion f : D U, and is oupu is a value in U. We will wrie he summaion as D f(), or jus f(), where f is an expression and is a free variable. Unbounded summaion furher saisfies he following axioms: ( f1() + f ) 2() = f 1() + f 2() (7) f( 1, 2) = f( 1, 2) (8) x f 2() = x f 2() (9) f() = f() (10) Thus, a U-semiring exends he semiring wih unbounded summaion, he squash operaor (o model he SQL DISTINCT operaor), and negaion (o model NOT EXISTS). We chose a se of axioms ha capures he semanics of SQL queries. For example, Eq.(2) implies x = x, which, as we will see, helps us prove ha DISTINCT of DISTINCT equals DISTINCT. Eq.(4) capures equaliy of queries under se semanics, such as SELECT DISTINCT R.a FROM R and SELECT DISTINCT x.a FROM R x, R y WHERE x.a = y.a. Eq.(6) capures even more suble ineracions beween subqueries wih and wihou DISTINCT as described in [44]. We will illusrae his in Sec If a summaion domain D is finie, say D = {a 1,..., a n}, we could define Σ Df direcly as f(a 1)+ +f(a n). However, as we saw, he meaning of a projecion requires us o sum over an unspecified se, namely, all uples of a fixed schema. Even hough all SQL summaion domains are finie, proving his auomaically is difficul and adds considerable complexiy even for he simples query equivalences (for insance, we need o prove ha all operaors preserve domain finieness). Insead, we reain unbounded summaion as a primiive in a U-semiring. We now illusrae four simple examples of U-semirings. (1) If all summaion domains are finie, hen he se of naural numbers, N, forms a U-semiring, where he unbounded summaion is he sandard sum, he squash and negaion operaors are 0 = no(x) = 0, and x = no(0) = 1 for all x 0. (2) Is closure, N = N { }, forms a U-semiring over arbirary summaion def domains. 4 (3) The univalen ypes [50] form an U-semiring; in our prior sysem, COSETTE [23], we used univalen ypes o prove SQL query equivalence. (4) Finally, he cardinal numbers (which form a subse of univalen ypes) also form a U-semiring. To appreciae he design of U-semirings, i may be helpful o see wha we excluded. Recall ha fewer axioms ranslae ino a 4 + and are exended by x + =, 0 = 0, and x = for x 0. Unbounded summaion is defined as D f def = f(a 1 ) + + f(a n) when he suppor of f is finie, supp(f) def = {x f(x) 0} = {a 1,..., a n}, and D f = oherwise. simpler proof sysem; hence, he need for frugaliy. A compleesemiring [30] also exends a semiring wih unbounded summaion. However, i requires a sronger se of axioms: summaion mus be defined over all subses of some index se, and includes addiional axioms, such as {a,b} f = f(a) + f(b), among ohers. In a U-semiring, summaion is defined on only a small and fixed se of summaion domains ha do no include {a, b}. This removes he need for axioms involving complex index ses. Similarly, he axioms for and no( ) are also kep o a minimum; for example, we omied unnecessary condiional ideniies like if x 0 hen x = 1. One example of a U-semiring where his condiional axiom fails is he se of diagonal 2 2 marices wih elemens in N, where 0 def = diag(0, 0), 1 def = diag(1, 1), and all operaions are performed on he diagonal using heir meaning in N. In his semiring, x {diag(0, 0), diag(0, 1), diag(1, 0), diag(1, 1)}. Anoher imporan decision was o exclude order relaions, x y, which we could have used o define key consrains (by saing ha every key value occurs 1 imes). An ordered semiring (see also a dioid [33]) is a semiring equipped wih an order relaion such ha 0 a for all a, and x y implies x + z y + z. We did no require a U-semiring o be ordered, insead we define key consrains using only he exising axioms (Sec. 4). Wih all hese simplificaions, one wonders if i is possible o prove any non-rivial SQL equivalences, le alone hose in he presence of complex inegriy consrains. We answer his in he affirmaive, as we shall nex explain. 3.2 U-semiring SQL Semanics Every SQL query is ranslaed ino a U-expression, which denoes he K-relaion of he query s answer; o check he equivalences of wo SQL queries, he sysem firs ranslaes hem o U- expressions, hen checks heir equivalence using he UDP algorihm described in Sec. 5. In his secion, we describe he ranslaion from SQL queries o U-expressions. Fig. 2 shows he SQL fragmen currenly suppored by our implemenaion. We require he explici declaraion of able schemas, keys, foreign keys, views, and indexes, and suppor a rich fragmen of SQL ha includes subqueries and DISTINCT. We also suppor GROUP BY by de-sugaring hem ino subqueries as follows: SELECT x.k as k, agg (x.a) as a1 FROM R x GROUP BY x.k is rewrien o SELECT y.k as k, agg ( SELECT x.a as a FROM R x WHERE x.k=y.k) as a1 FROM R y where R can be any SQL expression. Our implemenaion of UDP currenly suppors aggregaes by reaing hem as uninerpreed funcions. Each SQL query Q is ranslaed ino a U-expression as follows. We denoe a U-expressions by a capial leer E, denoe a SQL expression by a lower case leer e (Expression in Fig. 2, e.g.,.price/100), and denoe a SQL predicae by b (Predicae in Fig. 2, e.g..price/100 > s.discoun). For each schema σ in he program, here is a summaion domain Tuple(σ), which includes all uples wih schema σ. For each relaion name R, here is a predefined funcion R : Tuple(σ) U. Inuiively, R () reurns he mulipliciy of R, or 0 if R. If he conex is clear, we will drop and simply wrie R(). Unlike prior work [35], we do no require he suppor of R o be finie, as ha would make i difficul o axiomaize as discussed. 1485

5 For each SQL predicae expression b, here is a U-expression [b] U ha saisfies: [b] = [b] (11) Under he sandard inerpreaion of SQL, [b] is eiher 0 (false) or 1 (rue), bu formally we only require Eq.(11). The equaliy predicae has a special inerpreaion in U-semiring and is required o saisfy he following axioms, called excluded middle (Eq. (12)), subsiuion of equals by equals (Eq. (13)), and uniqueness of equaliy (Eq. (14)): [e 1 = e 2] + [e 1 e 2] =1 (12) f(e 1) [e 1 = e 2] =f(e 2) [e 1 = e 2] (13) [ = e] =1 (14) These axioms are sufficien o capure he semanics of equaliy (=). For example, we can prove: 5 [ = e] f() = f(e) (15) For each uninerpreed aggregae operaor agg and U-expression E : Tuple(σ) U, e ::= agg(e) is a valid value expression. I can be used in a predicae, e.g., [agg(e) =.a]. Every SQL query q is ranslaed, inducively, o a U-expression denoed q. For example, a able R is ranslaed ino is predefined semanics R ; a SELECT-FROM-WHERE is ranslaed by generalizing K-relaion SQL semanics (shown in Sec. 2); predicaes are also ranslaed inducively: NOT, AND, OR become no( ),, +, while EXISTS q = q and NOT EXISTS q = no( q ). Noice ha he language includes nesed queries boh in he FROM and he WHERE clauses; heir ranslaion is based on sandard unnesing. We omi he deails and sae only he main propery: DEFINITION 3.2. Every SQL expression q in Fig. 2 is ranslaed ino a U-expression q : Tuple(σ) U. The ranslaion is defined inducively on he srucure of q; see he full paper for deails [20]. We wrie q insead of q when conex permis. Sricly speaking q is a funcion, q = λ.e, bu we use he more friendly noaion q() = E(), where E is an expression wih a free variable. 3.3 Sum-Produc Normal Form To faciliae he auomaed equivalence proof, our algorihm, UDP, firs convers every U-expression ino he Sum-Produc Normal Form (SPNF). Imporanly, his conversion is done by repeaed applicaions of he U-semiring axioms; hus, our sysem can prove ha any expression ranslaed ino SPNF is semanically equivalen o he original inpu U-expression. DEFINITION 3.3. A U-expression expression E is in SPNF if i has he following form: E ::= T T n Furhermore, each erm T i has he form: 1,..., m [b 1]... [b k ] E s no(e n) M 1... M j where each uple variable i ranges over Tuple(σ i). Each expression inside he summaion is called a facor. Muliple summaions are combined ino a single sum using Eq. (8). There is no summaion in he case where m = 0. 5 Using (13), 9, and (14) we can show ha: [ = e] f() = [ = e] f(e) = f(e) [ = e] = f(e) 1 = f(e). SQL Query q SELECT 2.* FROM ( SELECT 1.k as k, 1.a as a FROM R 3) 1, R 2 WHERE 1.k = 2.k AND 1.a 12 Is corresponding U-expression in SPNF q () = [ 2 = ] ( [ 3.k = 1.k] [ 3.a = 1.a] 1, 2 3 R( 3)) R( 2) [ 1.k = 2.k] [ 1.a 12] = [ 2 = ] [ 1.k = 2.k] [ 1.a 12] 1, 2 ( ) [ 3.k = 1.k] [ 3.a = 1.a] R( 3) R( 2) Rule (4) 3 = 1, 2 [ 2 = ] [ 1.k = 2.k] ([ 1.a 12] 3 [ 3.k = 1.k] [ 3.a = 1.a] R( 3) R( 2) Rule (7) = 1, 2, 3 [ 2 = ] [ 1.k = 2.k] [ 1.a 12] [ 3.k = 1.k] [ 3.a = 1.a] R( 3) R( 2) Rule (6) Figure 3: A SQL query q (he second query shown in Fig. 1), is semanics q in U-semiring and is rewriing ino sum-produc normal form. Each facor [b i] is a predicae. There is exacly one facor E s, where E s is an expression in SPNF. When E s = 1, E s can be omied. There is exacly one facor no(e n), where E n is an expression in SPNF. When E n = 0, no(e n) can be omied. Each facor M i is an expression of he form R(), for some relaion name R, and some uple variable. THEOREM 3.4. For any U-expression E, here exiss an SPNF expression E such ha E = E in any U-semiring. PROOF. We briefly skech he proof idea here. Formally, any U- expression E can be rewrien ino E in SPNF using he following rewrie sysem: E 1 (E 2 + E 3) E 1 E 2 + E 1 E 3 (1) (E 1 + E 2) E 3 E 1 E 3 + E 2 E 3 (2) E 1 (E 2 E 3) (E 1 E 2) E 3 (3)... M [b] [b] M... (4) ( f1() + f ) 2() f 1() + f 2() (5) E f() E f() (6) ( f() ) E f() E (7) E 1 E 2 E 1 E 2 (8) no(e 1) no(e 2) no(e 1 + E 2) (9) Each rule above corresponds o an axiom of U-semirings. Rule (1)- (4) are axioms of commuaive semirings, while he res are axioms of U-semirings. All rewrie rules in SPNF are unidirecional and guaraneed o make progress. For example Rule (1) and (5) apply disribuiviy o remove any + inside each T i. Rule (3) and (4) normalize he expressions so hey remain lef associaive wih all boolean expressions moved o he lef. The las wo rules, Rule 1486

6 (8) and (9), consolidae a produc of muliple facors ino a single facor. Figure 3 shows how a U-expression is convered ino SPNF by applying Rules (4, 7), and (6). In general, he rules described above are applied recursively unil none of hem is applicable. 4. INTEGRITY CONSTRAINTS Our sysem checks he equivalence of wo SQL queries in he presence of a se of inegriy consrains: keys, foreign keys, views, and indexes (see Fig. 2). In his secion we inroduce a new axiomaic inerpreaion of inegriy consrains using ideniies in a U-semiring. 4.1 Axiomaic Inerpreaion of Consrains Key Consrains. To he bes of our knowledge, key consrains have no o dae been defined for semiring semanics. Therefore, we give he following definiion. DEFINITION 4.1. Le R be a relaion wih schema σ, and le k be an aribue (or a se of aribues). The KEY consrain is he following ideniy, for all, Tuple(σ): [.k =.k] R() R( ) = [ = ] R() For each key consrain in he SQL specificaion he sysem generaes one such ideniy, adds i as an axiom, and uses i laer o prove equivalences of SQL expressions. We show wo simple properies implied by he key consrain. Firs, seing =, we have (R()) 2 = R() for every uple. Second, for wo uples such ha.k =.k, we have R() R( ) = 0. Therefore, if he U-semiring is he semiring of naural numbers, N, hen R.k is a sandard key: he firs propery implies ha R() {0, 1}, and he second implies ha R() = 0 or R( ) = 0. Hence, only one uple wih a given key may occur in R, wih is mulipliciy being 1: THEOREM 4.2. If R.k saisfies he key consrain (Def. 4.1) over he U-semiring of naural numbers N, hen R.k is a sandard key. We briefly discuss our choice in Def The sandard axiom for a key is,.( R R.k =.k = ) [11]. However, i applies only o se semanics and uses a firs-order senence insead of an ideniy in a semiring. An alernaive aemp a defining a key is: R() [.k = e] 1 (10) This says ha he sum of all mulipliciies of uples ha have heir keys equal o a consan e mus be 1. However, (10) requires an ordered semiring, which leads o addiional complexiy, as argued earlier. In conras, Def. 4.1 does no require order and is a simple ideniy, i.e., i has he form E 1 = E 2. A beer aemp is o sae: R( ) R() [.k = e] = R( ) (11) This axiom is an ideniy and, furhermore, i can be shown ha Eq.(10) implies Eq.(11) in any ordered semiring; in oher words, Eq.(11) seems o be he righ reformulaion of Eq.(10) wihou requiring order. On he oher hand, Eq.(11) already follows from our key ideniy (Def. 4.1): simply sum boh sides over and observe ha he RHS is equal o R( ). We prove here a consequence of he key consrain, which we use in Sec. 5.4 o prove some non-rivial SQL ideniies described by [44]: THEOREM 4.3. If R.k saisfies he key consrain (Def. 4.1), hen he following U-expression is preserved under he squash operaor, for any U-expression E, expression e, predicae b: [b] E [.k = e] R() = [b] E [.k = e] R() PROOF. By Eq.(6) in Sec. 3.1, i suffices o show ha he LHS squared equals iself. From Def. 4.1, we derive: 6 [.k = e] [.k = e] [b] 2 E 2 R() R( ) = [ = ] [.k = e] [b] E R() Nex, we sum over and on boh sides:, [.k = e] [.k = e] [b] 2 E 2 R() R( ) =, [ = ] [.k = e] [b] E R() By applying Eq. (9) in Sec. 3.1 wice on he lef, and Eq. (15) in Sec. 3.2 on he righ: ( ) ( ) [b] E [.k = e] R() [b] E [.k = e] R( ) = [b] E [.k = e] R() Hence Theorem 4.3 follows from Eq. (6) in Sec. 3.1: x 2 = x implies x = x. Foreign Key Consrains. We now define foreign keys in a U- semiring: DEFINITION 4.4. Le S, R be wo relaions, and k, k be wo aribues (or wo ses of aribues), in S and R, respecively. The FOREIGN KEY consrain from S.k o R.k is he following: S( ) = S( ) R() [.k =.k ] If S( ) 0 and he U-semiring is he sandard semiring N, hen his definiion implies R() [.k =.k ] = 1, i.e., a uple in R has he same value k, he uple is unique, and is mulipliciy is 1. There is no consrain on he mulipliciy of S( ). Thus: THEOREM 4.5. If S.k, R.k saisfies he foreign key consrain (Def. 4.4) over he U-semiring of naural numbers N, hen R.k is a sandard key in R, and S.k is a sandard foreign key o R.k. Views and Indexes. We conver a view definiion in Fig. 2 ino an asserion v() = q(); every occurrence of v in a query is inlined according o is definiion. We follow he GMAP approach [52] and consider an index o be a view definiion ha consiss of he projecion on he key and he aribue o be indexed. For example, an index I on he aribue R.a is expressed as: I := SELECT x.a, x.k FROM R x where k is he KEY (Def. 4.1) of R. The indexes are reaed as views and inlined in he main query when compiling o U-expressions. 6 We also used [b] 2 = [b] and E 2 = E. 1487

7 4.2 SQL Equivalence wih Inegriy Consrains We can now formally define he equivalence of wo SQL expressions under U-semiring semanics. DEFINITION 4.6. Consider wo queries, q 1 and q 2, along wih definiions of schemas, base relaions, and consrains (Fig. 2). q 1 and q 2 are U-equivalen if, for any U-semiring and inerpreaion of he base relaions, when all key and foreign key consrains are saisfied, hen he following ideniy holds: q 1 = q 2. 7 We will refer o U-equivalence simply as equivalence when he conex is clear. We illusrae his concep wih an example. EXAMPLE 4.7. We show how o formally prove ha he opimizaion rule a he op of Fig. 1 is correc by showing ha Q 1 is equivalen o is rewrie, Q 2 using an index lookup. Recall ha R has key k, and I is an index on R.a. The U-expression of Q 1 is: Q 1() = R() [.a 12] Afer inlining he definiion of I, Q 2 is he query shown in Fig.3, and is U-expression in SPNF is: Q 2() = [ 2 = ] [ 1.k = 2.k] [ 1.a 12] 1, 2, 3 [ 3.k = 1.k] [ 3.a = 1.a] R( 3) R( 2) Since 1 is a uple reurned by I, is schema consiss of he wo aribues a and k; hence, [ 3.k = 1.k] and [ 3.a = 1.a] imply [ 1 = ( 3.k, 3.a)]. We use Eq. (15) o remove he summaion over 1, he wo equaliies [ 3.k = 1.k] and [ 3.a = 1.a], and subsiue 1.k wih 3.k and 1.a wih 3.a o ge: Q 2 () = 2, 3 [ 2 = ] [ 3.k = 2.k] [ 3.a 12] R( 3 ) R( 2 ) Applying he key consrain definiion (Def. 4.1) on Q 2(), we ge: Q 2() = 2, 3 [ 2 = ] [ 3.a 12] R( 2) [ 2 = 3] Now, we use Eq. (15) o remove he summaion over 3 as 2 = 3: Q 2() = 2 [ 2 = ] [ 2.a 12] R( 2) And similarly remove he summaion over 2 since 2 = : Q 2() = [.a 12] R() This shows ha Q 1 and Q 2 are equivalen by Def The example above reveals a sraegy for proving queries: firs, express boh queries in SPNF. Then, repeaedly reduce he expression using he available consrains unil he hey become isomorphic and hence are equivalen. We formally presen our equivalence deciding algorihm in he nex secion. Discussion. Definiion 4.6 is sound because wo U-equivalen queries are also equivalen under he sandard inerpreaion in he semiring of naural numbers. However, he definiion is no complee: here exiss SQL queries ha are equivalen under he sandard semanics bu no U-equivalen. One such example consiss of queries ha are equivalen over all finie relaions bu no equivalen over infinie relaions. For example, here exiss senences ϕ in Firs Order Logic, called infiniy axioms, ha are always false when R is finie, bu can be saisfied by an infinie R; for example, ϕ may say R is non-empy; for every x occurring in R here 7 We assume all views are inlined ino q 1 and q 2 as discussed above. exiss a unique y such ha R(x, y) (i.e. R is a funcion); he mapping x y is injecive; and i is no surjecive. 8 Such a senence ϕ can be convered in SQL query Q ha reurns when ϕ is false, and reurns 1 when ϕ is rue. Then, on any finie daabase Q is equivalen o he empy-se query Q (e.g, wrien in SQL as selec disinc 1 from R where 0 0), bu Q, Q are no U-equivalen, because hey are disinc over he U-semiring N. 5. DECISION PROCEDURE FOR SQL We now presen UDP (U-expression Decision Procedure) for checking he equivalence of wo U-expressions wih consrains, where equivalence is he U-equivalence given by Definiion 4.6. As we are unaware of any heorem provers ha can auomaically reason abou he equivalences of U-expressions, we implemen our decision procedure using proof assisans, which ensures ha our procedure implemenaion is provably correc given he U-expression axioms ha we have defined. UDP suppors he SQL fragmen in Fig. 2. The inpu SQL queries are evaluaed under mixed se and bag semanics, which is he semanics ha mos real-world daabase sysems use. We show ha UDP is sound and also complee when he inpu are Union of Conjuncive Queries (UCQ) and evaluaed under se semanics only, or under bag semanics only. To he bes of our knowledge, his is he firs algorihm implemened ha can check he equivalence of UCQ wih inegriy consrains and evaluaed under se or bag semanics. A a high level, UDP proceeds recursively on he srucure of a U- expression as shown in Alg. 2. Recall from Def. 3.3 he srucure of a normalized U-expression E and of a erm T : E ::=T T n T i ::= [b 1 ]... [b k ] E no(e ) M 1... M j 1,..., m UDP akes wo U-expressions (E 1 and E 2), and a se of inegriy consrains (C) in he form of U-expression ideniies. I firs calls canonize (line 2) o ransform each U-expression ino a canonical represenaion (o be discussed Sec. 5.1). Then, UDP proceeds recursively on he srucure of he wo canonical U-expressions by calling TDP (line 7) shown in Alg. 3 o check he equivalence of each erm. Similarly, procedure TDP calls SDP (line 4) shown in Alg. 4 o check he equivalence of wo squashed U-expressions (Uexpressions in he form of E ). We discuss he deails below. 5.1 Canonical Form As we have illusraed in Ex 4.7, o prove he equivalence of SQL queries under inegriy consrains, we rewrie he inpu query U- expressions using he axiomaic inerpreaions of hese consrains as shown in Sec We call hese rewrien U-expressions he canonical form of U-expressions under inegriy consrains. Algorihm 1 shows he deail of canonizaion. To conver a U- expression E from SPNF o canonical form, we rewrie E by he axioms and definiions discussed in Sec. 3 and Sec. 4. The algorihm conains he following four rewries: 1. Apply he ransiiviy of he equaliy predicae (line 2): [e 1 = e 2] [e 2 = e 3] = [e 1 = e 2] [e 2 = e 3] [e 1 = e 3]. 2. Remove unnecessary summaions using Eq. (15) (line 3). 3. For each key consrain, rewrie E using he ideniies defined in Def. 4.1 and Theorem 4.3 (line 5). 8 A differen, shorer infiniy axiom is given in [15, pp.307]: ϕ x y z( R(x, x) R(x, y) (R(y, z) R(x, z))). 1488

8 Algorihm 1 Canonizaion 1: procedure canonize(e, C) // E is an U-expression in SPNF; C is a se of consrains 2: E TC(E) // Transiive closure of equaliies 3: E recursively and repeaedly apply Eq. (15) on E 4: for c C do 5: E recursively and repeaedly apply Def. 4.1 and Theorem 4.3 on E 6: E recursively and repeaedly apply Def. 4.4 on E 7: end for 8: reurn E // E is now in canonical form 9: end procedure 4. For each foreign key consrain, rewrie E using he ideniy defined in Def. 4.4 (line 6). When using an ideniy o rewrie E, we find a maching subexpression on E wih he ideniy s LHS, and hen replace i wih he RHS. More imporanly, hese rewries are performed recursively on he srucure of E, and repeaedly unil he following erminaion condiions: 1) The firs 3 rewries (line 2, 3, and 5) erminae unil no furher rewrie can be applied. 2) When applied on squashed expressions, e.g., E, he las rewrie (line 6) is repeaedly applied unil he rewrien squashed expression is equivalen o he one before applying he rewrie. The equivalence of he squashed expressions (before and afer applying he rewrie) is checked using procedure SDP ha will be presened in Algorihm 4 (Sec. 5.2). 3) When applied on oher expressions, he las rewrie (line 6) is repeaedly applied unil no new base relaion is inroduced. Does canonize erminae? The rewries using key and foreign key definiions resemble he chase procedure [45]. The difference is ha our rewries are on U-expressions raher han on relaional queries. I is known ha he chase procedure may no erminae [31], and so may our rewrie seps; for example, a cycle in he key/foreignkey graph may lead o non-erminaion. While his is possible in heory, we did no encouner any case ha does no erminae, as our evaluaion in Sec. 6 shows. 5.2 Equivalence of U-expressions We now presen he algorihm for checking if wo U-expressions are equivalen afer convering hem ino canonical form. Algorihm 2 UDP: U-expression Decision Procedure 1: procedure UDP(E 1, E 2, C) // E 1 and E 2 are in SPNF 2: E 1 canonize(e 1, C), E 2 canonize(e 2, C) // E 1 is in he form of T 1, T 1,n, and // E 2 is in he form of T 2, T 2,m 3: if m n hen 4: reurn false 5: end if 6: for p P([T 1,1,..., T 1,n]) do 7: if i {1,..., n}, TDP(p[i], T 2,i, C) == rue hen 8: reurn rue 9: end if 10: end for 11: reurn false 12: end procedure Alg. 2 shows he deail of UDP for checking he equivalence of wo U-expressions. On line 2, UDP firs convers he wo inpu U- expressions (E 1 and E 2) o canonical forms under inegriy consrains as described in Sec Recall ha a U-expression is a sum of erms E = T T n. To check for U-equivalence of E 1 and E 2, UDP searches for an isomorphism beween each of heir consiuen erms T i in line I reurns false if he number of erms of E 1 and E 2 are no equal (line 4). Oherwise, i searches for a permuaion p of E 1 s erms (here P represens he se of all possible permuaions of is argumens) such ha each pair of erms in p and E 2 are equivalen. This is deermined by calling TDP in line 8, which we discuss nex. Equivalence of Terms The TDP procedure shown in Alg. 3 checks he equivalence beween erms. Recall ha a erm is an unbounded summaion of he form T = 1,..., m (...). To check for he equivalence of he following inpu erms: T 1 = #» 1 [b 1,1]... [b 1,k ] E 1,1 no(e 1,2) M 1,1... M 1,j T 2 = #» 2 [b 2,1]... [b 2,m] E 2,1 no(e 2,2) M 2,1... M 2,l TDP searches for an isomorphism beween T 1 and T 2. Le h BI #» 2, #» 1 be a bijecion from he se of variables ha T 2 sums over ( #» 2) o he se of variables ha T 1 sums over ( #» 1) (line 2). 9 TDP find an isomorphism if, afer subsiuing #» 1 wih p( #» 2) in T 2 (we call his new expression T 2) (line 3), he equivalence of T 1 and T 2 is checked as following (line 4): The predicae pars of wo erms are equivalen: [b 1,1]... [b 1,k ] = [b 2,1]... [b 2,m]. This requires checking ha wo Boolean expressions are equivalen. We check he equivalences of boolean expressions using he congruence procedure [43], which firs compues he equivalen classes of variables and funcion applicaions and hen checks for equivalence of he expressions using he equivalen classes. For example, [a = b] [c = d] [b = e] [f(a) = g(d)] is equivalen o [a = b] [a = e] [c = d] [f(e) = g(c)], since here are he following equivalen classes: {a, b, e}, {c, d}, {f(a), f(e)}, {g(c), g(d)} A predicae in T 1 (e.g., [b 1,i]) may conains aggregae funcions (for example, [avg( ) =.a]). These aggregae funcions are reaed as uninerpreed funcions like f and g in he example above. E 1,1 is U-equivalen o E 2,1. We explain he procedure for checking equivalences of squashed U-expressions, SDP, in he nex secion. The negaed expressions no(e 1,2) and no(e 2,2) are U-equivalen. This is checked by calling UDP recursively in line 4. The wo erms have he same number of M suberms, i.e., j = l, and he erms M 1,1,..., M 1,j are idenical o he erms M 2,1,..., M 2,l. Recall ha each M i1,i 2 has he form R() for some relaion name R and some uple variable (Def. 3.3). Equivalences of Squashed Expressions Finally, Alg. 4 shows SDP, he procedure for checking he equivalence of wo squashed U-expressions, E 1 and E 2. Recall ha squash expressions are used o model he semanics of (sub-)queries wih he DISTINCT operaor. The firs sep of SDP is o remove any nesed squash subexpressions (line 2). We do so by applying he following lemma: 9 For ease of presenaion, we use he vecorized noaion #» as a shorhand for 1, 2,

9 Algorihm 3 TDP: Decision Procedure for Terms 1: procedure TDP(T 1, T 2, C) // T 1, T 2 is in SPNF, C is a se of consrains. In paricular, // T 1 has he form #» 1 [b 1,1]... [b 1,k ] E 1,1 // no(e 1,2) M 1,1... M 1,j, and // T 2 has he form #» 2 [b 2,1]... [b 2,m] E 2,1 // no(e 2,2) M 2,1... M 2,l 2: for p BI #» 2, #» 1 do 3: T 2 p(t 2) // subsiue #» 2 in T 2 using p( #» 2) 4: if T 1 == T 2 hen 5: reurn rue 6: end if 7: end for 8: reurn false 9: end procedure LEMMA 5.1. The following holds in any U-semiring: a x + y = a x + y (12) PROOF. By Eq. (2) in Sec. 3, LHS = a x + y. And by Eq. (3), LHS = a x + y. By Eq. (2) (when y = 0 in Eq. (2)), x = x; hus: Apply Eq. (3) again: LHS = a x + y LHS = a x + y And apply Eq. (2) from righ o lef: LHS = a x + y Algorihm 4 SDP: Decision Procedure for Squashed Expressions 1: procedure SDP( E 1, E 2, C) // E, E are in SPNF 2: remove inside E 1, E 2 by applying Lem : E 1 canonize(e 1, C) // E 1 is now in he form of T T m 4: E 2 canonize(e 2, C) 5: // E 2 is now in he form of T T n reurn i j. minimize( T i ) == minimize( T j ) && j i. minimize( T j ) == minimize( Ti ) 6: end procedure SDP hen convers he expressions E 1 and E 2 o canonical forms under consrains by calling canonize on (lines 3-4). Afer ha, SDP checks he equivalence of wo expressions T T m and T T m. If T i and T i (for i = 1,..., m) are conjuncive queries, hen T T m and T T m represen wo queries in he class of unions of conjuncive queries under se semanics. A classical algorihm exiss for checking he equivalence of such queries [47]: given Q = q 1 q m and Q = q 1 q n, where each q is a conjuncive query, equivalence is esablished by checking wheher Q Q and Q Q. The former is checked by showing ha for every i, here exiss a j such ha q i q j which, in urn, requires checking for a homomorphism from q j o q i. Q Q is checked similarly. However, i is challenging o express he homomorphism checking algorihm in [47] purely using U-semiring axioms since U- semiring axioms are all ideniies (equaions). There is no noion of inclusion or ordering in U-semiring. To check he equivalences of wo canonized squashed expressions, E 1 and E 2, we insead minimize each erm, T i, inside E 1 and E 2 using only he U-semiring axioms (his is implemened in he minimize procedure) and hen check he synacic equivalences of he minimized erms. This procedure is equivalen o minimizing conjuncive queries [11]. Thus, our equivalence checking procedure is sound and complee for squashed expressions derived from UCQs. For squashed expressions ha represen SQL queries beyond UCQs, SDP is no complee, however, our procedure is sill sound. Nex, we explain how o minimize a erm in a squashed expression using only U-semiring axioms. minimize follows exacly he same seps as shown in he example, excep needs o be performed on all possible pairs of summaion variables. EXAMPLE 5.2. We illusrae SDP on hese wo queries: SELECT DISTINCT x.a FROM R x, R y -- Q1 SELECT DISTINCT R.a FROM R -- Q2 I firs convers Q 1 o a U-expressions canonical form (line 2-4): Q 1 () = [ 1.a = ] R( 1 ) R( 2 ) 1, 2 Eq.((12)) = ([ 1 = 2 ] + [ 1 2 ]) [ 1.a = ] R( 1 ) R( 2 ) 1, 2 Nex, i uses he rewrie rules in Sec. 3.3 o conver i o SPNF: Q 1() = 1, 2 [ 1 = 2] [ 1.a = ] R( 1) R( 2)+ 1, 2 [ 1 2] [ 1.a = ] R( 1) R( 2) Using Eq. (15), i removes he summaion over 2 as 1 = 2: Q 1() = 1 [ 1.a = ] R( 1) R( 1)+ 1, 2 [ 1 2] [ 1.a = ] R( 1) R( 2) By Eq. (10) and Eq. (4) ( R( 1) R( 1) = R( 1) ), we have: Q 1() = 1 [ 1.a = ] R( 1)+ 1, 2 [ 1 2] [ 1.a = ] R( 1) R( 2) Now, we can facorize he expression inside squash using Eq. (7): Q 1() = 1 [ 1.a = ] R( 1)(1 + 2 [ 1 2] R( 2) Finally, we simplify Q 1() using Eq. (1) and Eq. (3): Q 1() = 1 [ 1.a = ] R( 1) 1 = 1 [ 1.a = ] R( 1) = Q 2() hus proving ha Q 1() and Q 2() are equivalen. 1490

10 5.3 Soundness and Compleeness We now show ha UDP, our procedure for checking he equivalence of wo U-expressions, is sound. Furhermore, i is complee if wo U-expressions are unions of conjuncive queries evaluaed under se or bag semanics. THEOREM 5.3. Algorihm 2 is sound. For any pair of SQL queries, if algorihm 2 reurns rue, hen he pair is equivalen under he sandard SQL semanics [25]. PROOF. All ransformaions in algorihms 1 and 2 are based on axioms of U-semiring and proven ideniies. Since he sandard SQL semanics [25] is based on he semiring of naural numbers N, i follows ha he equivalence also holds under he sandard semanics. When here is no inegriy consrain and Q is a CQ evaluaed under bag semanics, i.e., Q has he form SELECT p FROM R1 1,..., Rn n WHERE b, and b is a conjuncion of equaliy predicaes, hen Q has a unique canonical form, namely canonize( Q, ) = [b] R 1( 1)... R n( n) 1,..., n In addiion, if Q is a CQ evaluaed under se semanics, i.e., Q has he form SELECT DISTINCT p FROM R1 1,..., Rn n WHERE b, hen Q has a similar unique canonical form (he same form as bag semanics CQ bu wihin ). This allows us o prove he following wo heorems on he compleeness of UDP. THEOREM 5.4. Algorihm 2 is complee for checking he equivalence of Unions of Conjuncive Queries (UCQ) evaluaed under bag semanics. PROOF. Two UCQ queries under bag semanics are equivalen if and only if hey are isomorphic [47] (see also [24, Theorem 4.3, 4.4]), implying ha our algorihm is complee in his case. THEOREM 5.5. Algorihm 2 is complee for checking he equivalence of Unions of Conjuncive Queries (UCQ) evaluaed under se semanics. PROOF. The U-expression of a conjuncive query evaluaed under se semanics is [b1] [bn] R1(1) Rj(j), where all predicaes b i are equaliies [.a 1 =.a 2]. In his case, UDP simply checks for he exisence of a homomorphism for he inpu queries, which has been shown o be complee [47]. 5.4 An Illusraion We demonsrae how UDP works using an example rewrie evaluaed under mixed se-bag semanics from he Sarburs opimizer [44]: -- Q1 SELECT ip.np, im. ype, im. iemno FROM ( SELECT DISTINCT ip. iemno as in, ip. np as np FROM price price WHERE price. np > 1000) ip, im im WHERE ip. in = im. iemno ; -- Q2 SELECT DISTINCT price.np, im. ype, im. iemno FROM price price, im im WHERE price. np > 1000 AND ip. iemno = im. iemno ; Here iemno is a key of im. Below is he U-expression represening Q 1(): Q 1 () = 1, 2 [ 1.np =.np] [ 2.ype =.ype] [ 2.iemno =.iemno] [ 1.in = 2.iemno] [.iemno = 1.in] [.np = 1.np] [.np > 1000] price( ) im( 2 ) Nex, Q 1() is canonized by canonize (called by UDP in line 2). As uple 1 is generaed by he subquery in Q1, i has wo aribues: np and in. Because [ 1.np =.np)] and [ 1.in = 2.iemno], he summaion on 1 is removed afer applying Eq. (15) in line 3 in canonize: Q 1 () = 2 [ 2.ype =.ype] [ 2.iemno =.iemno] [.iemno =.iemno] [.np =.np] [.np > 1000] price( ) im( 2 ) Since ie.iemno is a key, Theorem 4.3 is applied (line 5 in canonize): Q 1 () = 2, [ 2.ype =.ype] [ 2.iemno =.iemno] [.iemno =.iemno] [.np =.np] [.np > 1000] price( ) im( 2 ) Similarly, Q 2() is canonized o: Q 2 () = 1, 2 [ 2.ype =.ype] [ 2.iemno =.iemno] [ 1.iemno =.iemno] [ 2.iemno = 1.iemno] [ 1.np =.np] [ 1.np > 1000] price( 1 ) im( 2 ) A he end, since Q 1() and Q 2() are squashed expressions, SDP is called. SDP finds a homomorphism from Q 2() o Q 1(), namely { 1, 2 2} and a homomorphism from Q 1() o Q 2() defined by he funcion { 1, 2 2}. Hence Q 1 is equivalen o Q 2. To he bes of our knowledge, his is he firs ime ha his rewrie rule is formally proved o be correc. By modeling he semanics of SQL using U-expressions, our procedure can be used o auomaically deduce he equivalence of complex rewries. 6. EVALUATION In his secion we describe our implemenaion and evaluaion of UDP. We firs describe our implemenaion in Sec Then, in Sec. 6.2, we repor on he wo ses of query rewrie rules or query pairs ha we used in he evaluaion: one se from well-known daa managemen research lieraure, and anoher from a popular open-source query opimizaion framework called Calcie [1]. We summarize he evaluaion resuls of our query equivalence checking algorihm in Sec. 6.2 and characerize hese resuls in Sec Sec. 6.4 concludes wih he limiaions of our curren implemenaion. 6.1 Implemenaion We have implemened our UDP equivalence checking algorihm, and Fig. 4 shows is archiecure. As shown in he figure, Our implemenaion akes as inpu a pair of SQL queries o be checked (Q 1 and Q 2) and he precondiion for hese queries in he form of inegriy consrains (C). I compiles he queries o U-expressions (E 1 and E 2) and checks he equivalence of he resuling U-expressions using UDP. We implemened UDP in Lean, a proof assisan, which guaranees ha if UDP reurns rue hen he inpu queries are indeed equivalen according o our U-semiring semanics. UDP is implemened in wo componens: a U-expression generaor ha parses he inpu SQL queries (expressed using he synax shown in Fig. 2) o ASTs (Absrac Synax Trees), and a converer ha ranslaes he ASTs o U-expressions. The parser is wrien in 440 lines of Haskell, and he converer is wrien using 202 lines of Lean. In addiion, he implemenaion of he axioms discussed 1491