Can We Trust SQL as a Data Analytics Tool?

Transcription

1 Can We Trust SQL as a Data nalytics Tool?

2 SQL The query language for relational databases International Standard since 1987 Implemented in all systems (free and commercial) $30B/year business Most common tool used by data scientists

3 Main Questions Do we understand SQL queries, even simple ones? nd if we think we do, do query results make sense?

4 sking these questions now? bit of history: before 1969, various ad-hoc database modes (network, hierarchical) writing queries: a very elaborate task ll changed in 1969: Codd s relational model; now dominates the world. Query writing made easy: SQL

5 Relational Model Orders Pay Customer ORDER_ID TITLE PRICE Ord1 Big Data 30 Ord2 SQL 35 Ord3 Logic 50 CUST_ID c1 c2 ORDER Ord1 Ord2 CUST_ID c1 c2 NME John Mary

6 Relational Model Orders Pay Customer ORDER_ID TITLE PRICE Ord1 Big Data 30 Ord2 SQL 35 Ord3 Logic 50 CUST_ID c1 c2 ORDER Ord1 Ord2 CUST_ID c1 c2 NME John Mary Language: Relational lgebra (R) projection π (find book titles) selection σ (find books that cost at least 40) Cartesian product union difference -

7 Queries Find ids of customers who buy all books: πcust_id (Pay) - πcust_id ((π cust_id(pay) πtitle(order)) - πcust_id,title (σorder_id=order (Order Pay)))

8 Queries Find ids of customers who buy all books: πcust_id (Pay) - πcust_id ((π cust_id(pay) πtitle(order)) - πcust_id,title (σorder_id=order (Order Pay))) That s not pretty. But here is a better idea (1971): express queries in logic

9 Queries Find ids of customers who buy all books: πcust_id (Pay) - πcust_id ((π cust_id(pay) πtitle(order)) - πcust_id,title (σorder_id=order (Order Pay))) That s not pretty. But here is a better idea (1971): express queries in logic {c (o,t,p) Order (o,t,p ) Order: (c,o ) Pay}

10 Queries Find ids of customers who buy all books: πcust_id (Pay) - πcust_id ((π cust_id(pay) πtitle(order)) - πcust_id,title (σorder_id=order (Order Pay))) That s not pretty. But here is a better idea (1971): express queries in logic {c (o,t,p) Order (o,t,p ) Order: (c,o ) Pay} This is first-order logic (FO). Codd 1971: R = FO.

11 History continued Of course programmers don t write logical sentences, they need a programming syntax. Enters SQL: SELECT P.cust_id FROM P WHERE NOT EXISTS (SELECT * FROM Order O WHERE NOT EXISTS (SELECT * FROM Order O1 WHERE O1.title=O.title ND O1.order_id=P.order))

12 History continued Of course programmers don t write logical sentences, they need a programming syntax. Enters SQL: SELECT P.cust_id FROM P WHERE NOT EXISTS xf(x) = (SELECT * FROM Order O x F(x) WHERE NOT EXISTS (SELECT * FROM Order O1 WHERE O1.title=O.title ND O1.order_id=P.order))

13 History continued Of course programmers don t write logical sentences, they need a programming syntax. Enters SQL: SELECT P.cust_id FROM P WHERE NOT EXISTS (SELECT * FROM Order O WHERE NOT EXISTS (SELECT * FROM Order O1 WHERE O1.title=O.title ND O1.order_id=P.order)) xf(x) = x F(x) Take FO and turn into into programming syntax: Committee design! Then use R to implement queries.

14 SQL development Standards: SQL-86, SQL-89, SQL-92, SQL:1999, SQL:2003, SQL:2008, SQL:2011, SQL:2016 The latest standard will make you $1000 poorer The core remains the same. nd yet things are not as obvious as they should be. Now a few quiz-type slides.

15 TSK: Relations R(), S() Compute R - S.

16 TSK: Relations R(), S() Compute R - S. Every student will write: select R. from R where R. not in (select S. from S)

17 TSK: Relations R(), S() Compute R - S. Every student will write: select R. from R where R. not in (select S. from S) nd they are taught it is equivalent to : select R. from R where not exists (select S. from S where S.=R.)

18 TSK: Relations R(), S() Compute R - S. Every student will write: select R. from R where R. not in (select S. from S) nd they are taught it is equivalent to : select R. from R where not exists (select S. from S where S.=R.) and that they can do it directly in SQL: select * from r except select * from s

19 TSK: Relations R(), S() Compute R - S. R 1 null S null Every student will write: select R. from R where R. not in (select S. from S) nd they are taught it is equivalent to : select R. from R where not exists (select S. from S where S.=R.) and that they can do it directly in SQL: select * from r except select * from s

20 TSK: Relations R(), S() Compute R - S. R 1 null S null Outputs: Every student will write: select R. from R where R. not in (select S. from S) nd they are taught it is equivalent to : select R. from R where not exists (select S. from S where S.=R.) and that they can do it directly in SQL: select * from r except select * from s

21 TSK: Relations R(), S() Compute R - S. R 1 null S null Outputs: Every student will write: select R. from R where R. not in (select S. from S) nd they are taught it is equivalent to : select R. from R where not exists (select S. from S where S.=R.) and that they can do it directly in SQL: select * from r except select * from s

22 TSK: Relations R(), S() Compute R - S. R 1 null S null Outputs: Every student will write: select R. from R where R. not in (select S. from S) nd they are taught it is equivalent to : 1 null select R. from R where not exists (select S. from S where S.=R.) and that they can do it directly in SQL: select * from r except select * from s

23 TSK: Relations R(), S() Compute R - S. R 1 null S null Outputs: Every student will write: select R. from R where R. not in (select S. from S) nd they are taught it is equivalent to : 1 null select R. from R where not exists (select S. from S where S.=R.) and that they can do it directly in SQL: select * from r except select * from s 1

24 SELECT * should be simple, no? R Q = SELECT R., R. FROM R on 1 gives 1 1

25 SELECT * should be simple, no? R Q = SELECT R., R. FROM R on 1 gives 1 1 Let s use it as a subquery: Q = SELECT * FROM (Q) S T

26 SELECT * should be simple, no? R Q = SELECT R., R. FROM R on 1 gives 1 1 Let s use it as a subquery: Q = SELECT * FROM (Q) S T Output: Postgres: as above Oracle, MS SQL Server: compile-time error

27 SELECT * should be simple, no? R Q = SELECT R., R. FROM R on 1 gives 1 1 Let s use it as a subquery: Q = SELECT * FROM (Q) S T Output: Postgres: as above Oracle, MS SQL Server: compile-time error SELECT R. FROM R WHERE EXISTS (Q )

28 SELECT * should be simple, no? R Q = SELECT R., R. FROM R on 1 gives 1 1 Let s use it as a subquery: Q = SELECT * FROM (Q) S T Output: Postgres: as above Oracle, MS SQL Server: compile-time error SELECT R. FROM R WHERE EXISTS (Q ) nswer: 1

29 SELECT * should be simple, no? R Q = SELECT R., R. FROM R on 1 gives 1 1 Let s use it as a subquery: Q = SELECT * FROM (Q) S T Output: Postgres: as above Oracle, MS SQL Server: compile-time error SELECT R. FROM R WHERE EXISTS (Q ) nswer: 1 Except in MySQL

30 nother example: Query equivalences Q1(x) :- T(x,y) Q2(x) :- T(x,y), T(u,v)

31 nother example: Query equivalences Q1(x) :- T(x,y) Q2(x) :- T(x,y), T(u,v) In theory: equivalent; on B return 1 3

32 nother example: Query equivalences Q1(x) :- T(x,y) Q2(x) :- T(x,y), T(u,v) In theory: equivalent; on B return 1 3 Now the same in SQL:

33 nother example: Query equivalences Q1(x) :- T(x,y) Q2(x) :- T(x,y), T(u,v) In theory: equivalent; on B return 1 3 Now the same in SQL: Q1 = SELECT R. FROM R returns 1 3

34 nother example: Query equivalences Q1(x) :- T(x,y) Q2(x) :- T(x,y), T(u,v) In theory: equivalent; on B return 1 3 Now the same in SQL: Q1 = SELECT R. FROM R returns 1 3 Q2 = SELECT R1. FROM R R1, R R2 returns

35 Why do we find these questions difficult? Reason 1: there is no formal semantics of SQL. The Standard is rather vague, not written formally, and different vendors interpret it differently. Reason 2: theory works with a simplified model, no nulls, no duplicates, no repeated attributes. Under these assumptions several semantics exist ( ) but they do not model the real language.

36 It is much harder to deal with the real thing than with theoretical abstractions

37 It is much harder to deal with the real thing than with theoretical abstractions

38 From spherical to real cows We do it for the basic fragment of SQL: SELECT-FROM-WHERE without aggregation but with pretty much everything else [G.,L. Formal Semantics of SQL Queries, its Validation and pplications. PVLDB 2017]

39 Syntax : := T 1 S N 1,..., T k S N k for =(T 1,...,T k ), =(N 1,...,N k ), k > 0 : 0 := t 1 S N1, 0..., t m S Nk 0 for =(t 1,...,t m ), 0 =(N1,...,N 0 m), 0 m>0 Queries: Q := SELECT [ DISTINCT ] : 0 FROM : WHERE SELECT [ DISTINCT ] FROM : WHERE Q (UNION INTERSECT EXCEPT) [LL ] Q Conditions: := TRUE FLSE P (t 1,...,t k ),P2P t IS [ NOT ] NULL t [ NOT ] IN Q EXISTS Q ND OR NOT P Names: either simple (R, ) or composite (R.) Terms t: constants, nulls, or composite names Predicates: anything you want on constants

40 Semantics: labels `(R) = tuple of names provided by the schema `( ) =`(T 1 ) `(T k ) for =(T 1,...,T k ) SELECT [ DISTINCT ] : 0 ` FROM : WHERE = 0 ` SELECT [ DISTINCT ] FROM : WHERE = `( ) ` Q 1 (UNION INTERSECT EXCEPT) [LL ] Q 2 = `(Q 1 )

41 Semantics Q D,η,x Q: query D: database η: environment (values for composite names) x: Boolean switch to account for non-compositional nature of SELECT * (to show where we are in the query)

42 Semantics of terms JtK = J(t 1,...,t n )K = 8 >< () c >: NULL if t = if t = c 2 C if t = NULL Jt 1 K,...,Jt n K

43 Semantics: queries u s FROM : WHERE v SELECT : 0 FROM : WHERE u v SELECT FROM : WHERE u v SELECT FROM : WHERE s SELECT DISTINCT : 0 FROM : WHERE JRK D,,x = R D J : K D,,x = JT 1 K D,,0 JT k K D,,0 for =(T 1,...,T k ) 8 { < } ~ } ~ } ~ { D,,x D,,x D,,0 D,,1 D,,x = : 8 < = r,..., r {z } k times : J K 0,...,J K 0 {z } k times u } SELECT `( : ):`( ) = v FROM : ~ WHERE u = v SELECT c S N } FROM : ~ WHERE D,,1 s { SELECT : = " 0 FROM : WHERE r 2 k J : K D,,0, J K D, 0 = t, 0 = r `( : ) 0 = r `( : ), r 2 k s FROM : WHERE D,,0 for arbitrary c 2 C and N 2 N D,,x! { 9 = ; D,,x 9 = ;

44 Semantics: conditions JP (t 1,...,t k )K D, = Jt IS NULLK D, = 8 >< t if P Jt 1 K,...,Jt k K holds and Jt i K 6= NULL for all i 2{1,...,k} f if P Jt 1 K,...,Jt k K does not hold and Jt i K 6= NULL for all i 2{1,...,k} >: u if Jt i K = NULL for some i 2{1,...,k} ( t if JtK = NULL f if JtK 6= NULL Jt IS NOT NULLK D, = Jt IS NULLK D, n^ J(t 1,...t n )=(t 0 1,...,t 0 n)k D, = Jt i = t 0 ik D, J(t 1,...t n ) 6= (t 0 1,...,t 0 n)k D, = J t IN QK D, = i=1 8 >< t f >: u J t NOT IN QK D, = J t IN QK D, ( t if JQK D,,1 6=? JEXISTS QK D, = f if JQK D,,1 =? n_ Jt i 6= t 0 ik D, if 9 r 2 JQK D,,0 s.t. J t = rk D, = t if 8 r 2 JQK D,,0 s.t. J t = rk D, = f r 2 JQK D,,0 s.t. J t = rk D, = t and 9 r 2 JQK D,,0 s.t. J t = rk D, 6= f i=1 JTRUEK D, = t J 1 ND 2 K D, = J 1 K D, ^ J 2 K D, JNOT K D, = J K D, JFLSEK D, = f J 1 OR 2 K D, = J 1 K D, _ J 2 K D, Truth Tables: ^ t f u t t f u f f f f u u f u _ t f u t t t t f t f u u t u u t f u f t u

45 Semantics: operations JQ 1 UNION LL Q 2 K D,,x = JQ 1 K D,,0 [ JQ 2 K D,,0 JQ 1 INTERSECT LL Q 2 K D,,x = JQ 1 K D,,0 \ JQ 2 K D,,0 JQ 1 EXCEPT LL Q 2 K D,,x = JQ 1 K D,,0 JQ 2 K D,,0 JQ 1 UNION Q 2 K D,,x = " JQ 1 UNION LL Q 2 K D,,x JQ 1 INTERSECT Q 2 K D,,x = " JQ 1 INTERSECT LL Q 2 K D,,x JQ 1 EXCEPT Q 2 K D,,x = " JQ 1 K D,,0 JQ 2 K D,,0 Bag interpretation of operations; is duplicate elimination

46 Looks simple, no? It does not. Such basic things as variable binding changed several times till we got them right. The meaning of the new environment: FROM τ : β WHERE θ D,η,x = r,..., r }{{} k times r k τ : β D,η,0, θ D,η = t, η = η r l(τ : β) in η, unbind every name that occurs among labels of the FROM clause then bind non-repeated names among those to values taken from record r

47 How do we know we this is correct? Since the Standard is rather vague, there is only one way experiments. But what kind of benchmark can we use? For performance studies there are standard benchmarks like TPC-H. But they won t work for us: not enough queries.

48 Experimental Validation Benchmarks have rather few queries (22 in TPC-H). Validating on 22 queries is not a good evidence. But we can look at benchmarks, and then generate lots of queries that look the same. In TPC-H: 8 tables, maximum nesting depth = 3, average number of tables per query = 3.2, at most 8 conditions in WHERE (except two queries)

49 Validation: results Small adjustments of the Standard semantics (for Postgres and Oracle) Random query generator Naive implementation of the semantics Finally: experiments on 100,000 random queries

50 Validation: results Small adjustments of the Standard semantics (for Postgres and Oracle) Random query generator Naive implementation of the semantics Finally: experiments on 100,000 random queries Yes, it is correct!

51 What can we do with this? Equivalence of basic SQL and Relational lgebra: formally proved for the first time Previous attempts (Ceri and Gottlob, Van den Bussche and Vansummeren restricted the language severely: no nulls, for example). 3-valued logic of SQL vs the usual Boolean logic: 3-valued logic does not add expressiveness. lthough it does not mean we should get rid of it now

52 Does it matter which DBMS we use? We already saw it does. In fact in our experiments we adjusted things a bit for Postgres and Oracle. But how much of a difference does it make? We have a random query generator, so let s experiment: generate lots of queries (over 150K) send to standard DBMSs (Oracle, MySQL, MS SQL Server, PostgreSQL, IBM DB2) and see what happens

53 Discrepancies between RDBMSs bout 2% of queries do not behave the same way on different DBMSs and they come from the most basic fragment Lots of issues are minor and syntactic different syntax for set operations (eg EXCEPT vs MINUS) or functions (eg % vs MOD, or substring vs substr) But some are serious - and surprise even people with good SQL knowledge. Four of the most surprising examples to follow

54 Is empty string equal to itself? SELECT * FROM R WHERE =

55 Is empty string equal to itself? SELECT * FROM R WHERE = Usually it is, but not in Oracle: the above query always returns the empty table. Because Oracle implements NULL as Madness? Yes. With a string operation that produces you deal with 3-valued logic before you realize it!

56 Can you divide by zero? SELECT R./S.B FROM R, S R={1}, S={0}

57 Can you divide by zero? SELECT R./S.B FROM R, S R={1}, S={0} Usually not except in MySQL 5.6 It returns NULL OK, they realized it in MySQL 5.7 and now by default it s a warning. But one can go back to the 5.6 mode if one wishes

58 Is equality transitive? x=y and y=z imply x=z, right?

59 Is equality transitive? x=y and y=z imply x=z, right? Usually yes, but not in MySQL x= 1a, y=1, z= 1b Why is this a problem? SQL books teach programmers to overspecify join conditions: to R.=S. ND S.=T. add explicitly R.=T. But now it can turn a true condition into false!

60 Can you compare tuples in IN subqueries? SELECT * FROM R WHERE (R., R.B) IN SELECT (S., S.B FROM S)

61 Can you compare tuples in IN subqueries? SELECT * FROM R WHERE (R., R.B) IN SELECT (S., S.B FROM S) Usually yes, except in MS SQL Server. Why? No clue lso SQL Server has UNION but no UNION LL. Please explain this.

62 simple tool We actually have a tool that lets you: specify parameters of a query workload generate lots of random queries, and run against DBMSs you want to compare Have fun with results at least you know what to expect.