Modern SQL: Evolution of a dinosaur

Transcription

1 Modern SQL: Evolution of a dinosaur Markus Winand Kraków, 9-11 May 2018

2 Still using Windows 3.1? So why stick with -

3 SQL:1999

4 WITH (Common Table Expressions)

5 WITH (non-recursive) The Problem Nested queries are hard to read: SELECT FROM (SELECT FROM t1 JOIN (SELECT FROM Understand this first ) a ON ( ) ) b JOIN (SELECT FROM ) c ON ( )

6 WITH (non-recursive) The Problem Nested queries are hard to read: SELECT FROM (SELECT FROM t1 Then this... JOIN (SELECT FROM ) a ON ( ) ) b JOIN (SELECT FROM ) c ON ( )

7 WITH (non-recursive) The Problem Nested queries are hard to read: SELECT FROM (SELECT FROM t1 JOIN (SELECT FROM ) a ON ( ) ) b JOIN (SELECT FROM Then this... ) c ON ( )

8 WITH (non-recursive) The Problem Nested queries are hard to read: SELECT Finally the first line makes sense FROM (SELECT FROM t1 JOIN (SELECT FROM ) a ON ( ) ) b JOIN (SELECT FROM ) c ON ( )

9 WITH (non-recursive) Since SQL:1999 CTEs are statement-scoped views: Keyword WITH a (c1, c2, c3) AS (SELECT c1, c2, c3 FROM ), b (c4, ) AS (SELECT c4, FROM t1 JOIN a ON ( ) ),

10 WITH (non-recursive) Since SQL:1999 CTEs are statement-scoped views: WITH Name of CTE and (here optional) column names a (c1, c2, c3) AS (SELECT c1, c2, c3 FROM ), b (c4, ) AS (SELECT c4, FROM t1 JOIN a ON ( ) ),

11 WITH (non-recursive) Since SQL:1999 CTEs are statement-scoped views: WITH a (c1, c2, c3) Definition AS (SELECT c1, c2, c3 FROM ), b (c4, ) AS (SELECT c4, FROM t1 JOIN a ON ( ) ),

12 WITH (non-recursive) Since SQL:1999 CTEs are statement-scoped views: WITH a (c1, c2, c3) AS (SELECT c1, c2, c3 FROM ), Introduces another CTE b (c4, ) AS (SELECT c4, Don't repeat WITH FROM t1 JOIN a ON ( ) ),

13 WITH (non-recursive) Since SQL:1999 CTEs are statement-scoped views: WITH a (c1, c2, c3) AS (SELECT c1, c2, c3 FROM ), b (c4, ) AS (SELECT c4, FROM t1 JOIN a May refer to previous CTEs ON ( ) ),

14 WITH a (c1, c2, c3) WITH AS (SELECT c1, c2, c3 FROM ), (non-recursive) b (c4, ) AS (SELECT c4, FROM t1 JOIN a ON ( ) Since SQL:1999 ), Third CTE c ( ) AS (SELECT FROM ) SELECT FROM b JOIN c ON ( )

15 WITH a (c1, c2, c3) WITH AS (SELECT c1, c2, c3 FROM ), (non-recursive) b (c4, ) AS (SELECT c4, FROM t1 JOIN a ON ( ) ), c ( ) Since SQL:1999 AS (SELECT FROM ) No comma! SELECT FROM b JOIN c ON ( )

16 WITH a (c1, c2, c3) WITH AS (SELECT c1, c2, c3 FROM ), (non-recursive) b (c4, ) AS (SELECT c4, FROM t1 JOIN a ON ( ) ), c ( ) AS (SELECT FROM ) Since SQL:1999 SELECT Main query FROM b JOIN c ON ( )

17 WITH (non-recursive) CTEs are statement-scoped views: Since SQL:1999 WITH a (c1, c2, c3) AS (SELECT c1, c2, c3 FROM ), b (c4, ) AS (SELECT c4, FROM t1 JOIN a ON ( ) Read top down ), c ( ) AS (SELECT FROM ) SELECT FROM b JOIN c ON ( )

18 WITH (non-recursive) Use-Cases Literate SQL Organize SQL code to improve maintainability Assign column names to tables produced by values or unnest. Overload tables (for testing) with queries hide tables of the same name.

19 WITH (non-recursive) In a Nutshell WITH are the "private methods" of SQL WITH is a prefix to SELECT WITH queries are only visible in the SELECT they precede WITH in detail:

20 WITH (non-recursive) Availability MariaDB 8.0 MySQL 8.4 PostgreSQL [0] SQLite 7.0 DB2 LUW 9iR2 Oracle 2005 [1] SQL Server [0] Only for top-level SELECT statements [1] Only allowed at the very begin of a statement. E.g. WITH...INSERT...SELECT.

21 WITH RECURSIVE (Common Table Expressions)

22 WITH RECURSIVE The Problem (This page is intentionally left blank)

23 WITH RECURSIVE The Problem Coping with hierarchies in the Adjacency List Model [0] CREATE TABLE t ( id NUMERIC NOT NULL, parent_id NUMERIC, PRIMARY KEY (id) ) [0] Hierarchies implemented using a parent id see Joe Celko s Trees and Hierarchies in SQL for Smarties

24 WITH RECURSIVE The Problem Coping with hierarchies in the Adjacency List Model [0] SELECT * FROM t AS d0 WHERE LEFT JOIN d0.id t = AS? d1 ON (d1.parent_id=d0.id) LEFT JOIN t AS d2 ON (d2.parent_id=d1.id) [0] Hierarchies implemented using a parent id see Joe Celko s Trees and Hierarchies in SQL for Smarties

25 WITH RECURSIVE The Problem Coping with hierarchies in the Adjacency List Model [0] SELECT * FROM t AS d0 LEFT JOIN t AS d1 ON (d1.parent_id=d0.id) LEFT JOIN t AS d2 ON (d2.parent_id=d1.id) WHERE d0.id =? [0] Hierarchies implemented using a parent id see Joe Celko s Trees and Hierarchies in SQL for Smarties

26 WITH RECURSIVE Since SQL:1999 WITH RECURSIVE d (id, parent, ) AS SELECT * FROM t AS d0 LEFT JOIN t AS d1 ON (d1.parent_id=d0.id) LEFT JOIN t AS d2 ON (d2.parent_id=d1.id) WHERE d0.id =? (SELECT id, parent, FROM tbl WHERE id =? UNION ALL SELECT id, parent, FROM d JOIN tbl ON (tbl.parent=d.id) ) SELECT * FROM subtree

27 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: Keyword WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte

28 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte Column list mandatory here

29 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 Executed first UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte

30 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 Result sent there UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte

31 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) Result visible twice SELECT * FROM cte

32 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) Once it becomes part of the final result n SELECT * FROM cte 3 (3 rows)

33 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte Second leg of UNION is executed n (3 rows)

34 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte It's a loop! n (3 rows)

35 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte It's a loop! n (3 rows)

36 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte It's a loop! n (3 rows)

37 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte n=3 doesn't match n (3 rows)

38 WITH RECURSIVE Since SQL:1999 Recursive common table expressions may refer to themselves in a leg of a UNION [ALL]: WITH RECURSIVE cte (n) AS (SELECT 1 UNION ALL SELECT n+1 FROM cte WHERE n < 3) SELECT * FROM cte n=3 doesn't match Loop terminates n (3 rows)

39 WITH RECURSIVE Availability MariaDB 8.0 MySQL 8.4 PostgreSQL [0] SQLite 7.0 DB2 LUW 11gR2 Oracle [0] Only for top-level SELECT statements 2005 SQL Server

40 SQL:2003

41 OVER and PARTITION BY

42 OVER (PARTITION BY) The Problem Two distinct concepts could not be used independently: Merge rows with the same key properties GROUP BY to specify key properties DISTINCT to use full row as key Aggregate data from related rows Requires GROUP BY to segregate the rows COUNT, SUM, AVG, MIN, MAX to aggregate grouped rows

43 OVER (PARTITION BY) The Problem No Aggregate Yes Yes Merge rows No SELECT c1, c2 FROM t SELECT DISTINCT c1, c2 FROM t SELECT c1, SUM(c2) tot FROM t GROUP BY c1

44 OVER (PARTITION BY) The Problem No Aggregate Yes Yes Merge rows No SELECT c1, c2 FROM t SELECT DISTINCT c1, c2 FROM t SELECT c1 SELECT c1, c2, tot FROM t JOIN (, SUM(c2) tot FROM t SELECT c1, SUM(c2) tot FROM t GROUP BY c1 GROUP BY c1 ) ta ON (t.c1=ta.c1)

45 OVER (PARTITION BY) The Problem No Aggregate Yes Yes Merge rows No SELECT c1, c2 FROM t SELECT DISTINCT c1, c2 FROM t SELECT c1 SELECT c1, c2, tot FROM t JOIN (, SUM(c2) tot FROM t SELECT c1, SUM(c2) tot FROM t GROUP BY c1 GROUP BY c1 ) ta ON (t.c1=ta.c1)

46 OVER (PARTITION BY) Since SQL:2003 No Aggregate Yes Yes Merge rows No SELECT c1, c2 FROM t SELECT DISTINCT c1, c2 FROM t SELECT c1, c2 FROM, tsum(c2) OVER (PARTITION BY c1) FROM t SELECT c1, SUM(c2) tot FROM t GROUP BY c1

47 OVER (PARTITION BY) How it works dep salary ts SELECT dep, salary, SUM(salary) OVER() FROM emp

48 OVER (PARTITION BY) How it works dep salary ts SELECT dep, salary, SUM(salary) OVER() FROM emp

49 OVER (PARTITION BY) How it works dep salary ts SELECT dep, salary, SUM(salary) OVER() FROM emp

50 OVER (PARTITION BY) How it works dep salary ts SELECT dep, salary, SUM(salary) OVER() FROM emp

51 OVER (PARTITION BY) How it works dep salary ts SELECT dep, salary, SUM(salary) OVER() FROM emp

52 OVER (PARTITION BY) How it works dep salary ts SELECT dep, salary, SUM(salary) OVER() PARTITION BY dep) FROM emp

53 OVER and ORDER BY (Framing & Ranking)

54 OVER (ORDER BY) The Problem SELECT id, value, FROM transactions t acnt id value balance

55 OVER (ORDER BY) The Problem SELECT id, value, (SELECT SUM(value) FROM transactions t2 WHERE t2.id <= t.id) FROM transactions t acnt id value balance

56 OVER (ORDER BY) The Problem SELECT id, value, (SELECT SUM(value) FROM transactions t2 WHERE t2.id <= t.id) FROM transactions t Range segregation (<=) not possible with GROUP BY or PARTITION BY acnt id value balance

57 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( acnt id value balance ) FROM transactions t

58 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id acnt id value balance ) FROM transactions t

59 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING ) acnt id value balance FROM transactions t

60 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

61 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

62 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

63 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

64 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

65 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

66 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

67 OVER (ORDER BY) Since SQL:2003 SELECT id, value, SUM(value) OVER ( PARTITION BY acnt ORDER BY id ROWS BETWEEN UNBOUNDED PRECEDING acnt id value balance ) AND CURRENT ROW FROM transactions t

68 OVER (ORDER BY) Since SQL:2003 With OVER (ORDER BY n) a new type of functions make sense: n ROW_NUMBER RANK DENSE_RANK PERCENT_RANK CUME_DIST

69 OVER (SQL:2003) Use Cases Aggregates without GROUP BY Running totals, moving averages Ranking Top-N per Group Avoiding self-joins AVG( ) OVER(ORDER BY ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) moving_avg SELECT * FROM (SELECT ROW_NUMBER() OVER(PARTITION BY ORDER BY ) rn, t.* FROM t) numbered_t WHERE rn <= 3 [ many more ]

70 OVER (SQL:2003) In a Nutshell OVER may follow any aggregate function OVER defines which rows are visible at each row OVER() makes all rows visible at every row OVER(PARTITION BY ) segregates like GROUP BY OVER(ORDER BY BETWEEN) segregates using <, >

71 OVER (SQL:2003) Impala Availability Hive Spark NuoDB MariaDB 8.0 MySQL 8.4 PostgreSQL SQLite 7.0 DB2 LUW 8i Oracle 2005 SQL Server

72 SQL:2006

73 XMLTABLE

74 XMLTABLE Since SQL:2006 SELECT id Stored in tbl.x:, c1, n FROM tbl XPath * expression to identify rows <d> <e id="42"> <c1> </c1>, XMLTABLE( '/d/e' </e> </d> ) r PASSING x COLUMNS id INT PATH '@id', c1 VARCHAR(255) PATH 'c1', n FOR ORDINALITY *Standard SQL allows XQuery

75 XMLTABLE Since SQL:2006 SELECT id, c1, n FROM tbl, XMLTABLE( '/d/e' Stored in tbl.x: <d> <e id="42"> <c1> </c1> </e> </d> ) r PASSING x COLUMNS id INT PATH '@id', c1 VARCHAR(255) PATH 'c1', n FOR ORDINALITY *Standard SQL allows XQuery

76 XMLTABLE Since SQL:2006 SELECT id, c1, n FROM tbl, XMLTABLE( '/d/e' Stored in tbl.x: <d> <e id="42"> <c1> </c1> XPath * expressions </e> to extract data </d> ) r PASSING x COLUMNS id INT PATH '@id', c1 VARCHAR(255) PATH 'c1', n FOR ORDINALITY *Standard SQL allows XQuery

77 XMLTABLE Since SQL:2006 SELECT id, c1, n FROM tbl, XMLTABLE( '/d/e' Stored in tbl.x: <d> <e id="42"> <c1> </c1> </e> </d> PASSING x COLUMNS id INT PATH '@id' Row number (like for unnest), c1 VARCHAR(255) PATH 'c1', n FOR ORDINALITY ) r *Standard SQL allows XQuery

78 XMLTABLE Since SQL:2006 SELECT id Result Stored in tbl.x:, c1, n FROM tbl, XMLTABLE( '/d/e' PASSING x id c1 n <d> <e id="42"> <c1> </c1> </e> </d> ) r COLUMNS id INT PATH '@id', c1 VARCHAR(255) PATH 'c1', n FOR ORDINALITY *Standard SQL allows XQuery

79 XMLTABLE Availability MariaDB MySQL 10 [0] PostgreSQL SQLite 9.7 DB2 LUW 11gR1 Oracle SQL Server [0] No XQuery (only XPath). No default namespace declaration.

80 SQL:2008

81 FETCH FIRST

82 FETCH FIRST The Problem Limit the result to a number of rows. (LIMIT, TOP and ROWNUM are all proprietary) SELECT * FROM (SELECT *, ROW_NUMBER() OVER(ORDER BY x) rn FROM data) numbered_data WHERE rn <=10 SQL:2003 introduced ROW_NUMBER() to number rows. But this still requires wrapping to limit the result. And how about databases not supporting ROW_NUMBER()?

83 FETCH FIRST The Problem Limit the result to a number of rows. (LIMIT, TOP and ROWNUM are all proprietary) SELECT * Dammit! Let's take LIMIT FROM (SELECT * WHERE rn <=10, ROW_NUMBER() OVER(ORDER BY x) rn FROM data) numbered_data SQL:2003 introduced ROW_NUMBER() to number rows. But this still requires wrapping to limit the result. And how about databases not supporting ROW_NUMBER()?

84 FETCH FIRST Since SQL:2008 SQL:2008 introduced the FETCH FIRST ROWS ONLY clause: SELECT * FROM data ORDER BY x FETCH FIRST 10 ROWS ONLY

85 FETCH FIRST Availability MariaDB [0] MySQL 6.5 [1] 8.4 PostgreSQL [1] SQLite 7.0 DB2 LUW 12cR1 Oracle 7.0 [2] 2012 SQL Server [0] Earliest mention of LIMIT. Probably inherited from msql [1] Functionality available using LIMIT [2] SELECT TOP n... SQL Server 2000 also supports expressions and bind parameters

86 SQL:2011

87 OFFSET

88 OFFSET The Problem How to fetch the rows after a limit? (pagination anybody?) SELECT * FROM (SELECT *, ROW_NUMBER() OVER(ORDER BY x) rn FROM data) numbered_data WHERE rn > 10 and rn <= 20

89 OFFSET Since SQL:2011 SQL:2011 introduced OFFSET, unfortunately! SELECT * FROM data ORDER BY x OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY

90 OFFSET Since SQL:2011 SQL:2011 introduced OFFSET, unfortunately! OFFSET SELECT * FROM data ORDER BY x OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY Grab coasters & stickers!

91 OFFSET Since SQL: MariaDB [0] [1] MySQL 6.5 PostgreSQL SQLite 9.7 [2] 11.1 DB2 LUW 12c Oracle 2012 SQL Server [0] LIMIT [offset,] limit: "With this it's easy to do a poor man's next page/previous page WWW application." [1] The release notes say "Added PostgreSQL compatible LIMIT syntax" [2] Requires enabling the MySQL compatibility vector: db2set DB2_COMPATIBILITY_VECTOR=MYS

92 OVER

93 OVER (SQL:2011) The Problem Direct access of other rows of the same window is not possible. (E.g., calculate the difference to the previous rows) WITH numbered_t AS (SELECT * FROM t ) SELECT curr.*, curr.balance - COALESCE(prev.balance,0) curr balance rn 50 1 FROM numbered_t curr 90 2 LEFT JOIN numbered_t prev ON (curr.rn = prev.rn+1)

94 OVER (SQL:2011) The Problem Direct access of other rows of the same window is not possible. (E.g., calculate the difference to the previous rows) WITH numbered_t AS (SELECT *, ROW_NUMBER() OVER(ORDER BY x) rn FROM t) SELECT curr.*, curr.balance - COALESCE(prev.balance,0) curr balance rn 50 1 FROM numbered_t curr 90 2 LEFT JOIN numbered_t prev ON (curr.rn = prev.rn+1)

95 OVER (SQL:2011) The Problem Direct access of other rows of the same window is not possible. (E.g., calculate the difference to the previous rows) WITH numbered_t AS (SELECT *, ROW_NUMBER() OVER(ORDER BY x) rn FROM t) SELECT curr.*, curr.balance - COALESCE(prev.balance,0) curr balance rn 50 1 FROM numbered_t curr 90 2 LEFT JOIN numbered_t prev ON (curr.rn = prev.rn+1)

96 OVER (SQL:2011) The Problem Direct access of other rows of the same window is not possible. (E.g., calculate the difference to the previous rows) WITH numbered_t AS (SELECT *, ROW_NUMBER() OVER(ORDER BY x) rn FROM t) SELECT curr.*, curr.balance - COALESCE(prev.balance,0) curr balance rn 50 1 prev balance rn 50 1 FROM numbered_t curr LEFT JOIN numbered_t prev ON (curr.rn = prev.rn+1)

97 OVER (SQL:2011) The Problem Direct access of other rows of the same window is not possible. (E.g., calculate the difference to the previous rows) WITH numbered_t AS (SELECT *, ROW_NUMBER() OVER(ORDER BY x) rn FROM t) SELECT curr.*, curr.balance - COALESCE(prev.balance,0) curr balance rn 50 1 prev balance rn FROM numbered_t curr LEFT JOIN numbered_t prev ON (curr.rn = prev.rn+1)

98 OVER (SQL:2011) The Problem Direct access of other rows of the same window is not possible. (E.g., calculate the difference to the previous rows) WITH numbered_t AS (SELECT *, ROW_NUMBER() OVER(ORDER BY x) rn FROM t) SELECT curr.*, curr.balance curr balance rn prev balance rn - COALESCE(prev.balance,0) FROM numbered_t curr LEFT JOIN numbered_t prev ON (curr.rn = prev.rn+1)

99 OVER (SQL:2011) Since SQL:2011 SQL:2011 introduced LEAD, LAG, NTH_VALUE, for that: SELECT *, balance - COALESCE( LAG(balance) OVER(ORDER BY x), 0) FROM t Available functions: LEAD / LAG FIRST_VALUE / LAST_VALUE NTH_VALUE(col, n) FROM FIRST/LAST RESPECT/IGNORE NULLS

100 OVER (LEAD, LAG, ) Since SQL: [0] MariaDB 8.0 [0] MySQL 8.4 [0] PostgreSQL SQLite 9.5 [1] 11.1 DB2 LUW 8i [1] 11gR2 Oracle [0] No IGNORE NULLS and FROM LAST [1] No NTH_VALUE 2012 [1] SQL Server

101 System Versioning (Time Traveling)

102 System Versioning The Problem INSERT UPDATE DELETE are DESTRUCTIVE

103 System Versioning Since SQL:2011 Table can be system versioned, application versioned or both. CREATE TABLE t (..., start_ts TIMESTAMP(9) GENERATED ALWAYS AS ROW START, end_ts TIMESTAMP(9) GENERATED ALWAYS AS ROW END, PERIOD FOR SYSTEM_TIME (start_ts, end_ts) ) WITH SYSTEM VERSIONING

104 System Versioning Since SQL:2011 INSERT... (ID, DATA) VALUES (1, 'X') ID Data start_ts end_ts 1 X 10:00:00 UPDATE... SET DATA = 'Y'... ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 DELETE... WHERE ID = 1

105 System Versioning ID Data start_ts end_ts 1 X 10:00:00 Since SQL:2011 UPDATE... SET DATA = 'Y'... ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 DELETE... WHERE ID = 1 ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 12:00:00

106 System Versioning Since SQL:2011 ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 12:00:00 Although multiple versions exist, only the current one is visible per default. After 12:00:00, SELECT * FROM t doesn t return anything anymore.

107 System Versioning Since SQL:2011 ID Data start_ts end_ts 1 X 10:00:00 11:00:00 1 Y 11:00:00 12:00:00 With FOR AS OF you can query anything you like: SELECT * FROM t FOR SYSTEM_TIME AS OF TIMESTAMP ' :30:00' ID Data start_ts end_ts 1 X 10:00:00 11:00:00

108 System Versioning Since SQL: MariaDB [0] MySQL PostgreSQL SQLite 10.1 [1] DB2 LUW 10gR1 [2] Oracle 2016 SQL Server [0] Available in MariaDB 10.3 beta. [1] Third column required (tx id), history table required. [2] Functionality available using Flashback

109 SQL:2016 (released: )

110 LISTAGG

111 LISTAGG Since SQL:2016 grp val SELECT grp 1 B, LISTAGG(val, ', ') grp val 1 A WITHIN GROUP (ORDER BY val) 1 A, B, C 1 C FROM t 2 X GROUP BY grp 2 X Default LISTAGG(val, ', ' ON OVERFLOW ERROR) Default LISTAGG(val, ', ' ON OVERFLOW TRUNCATE '...' WITHOUT COUNT) 'A, B,...' LISTAGG(val, ', ' ON OVERFLOW TRUNCATE '...' WITH COUNT) 'A, B,...(1)'

112 LISTAGG Availability [0] MariaDB 4.1 [0] MySQL 7.4 [1] 8.4 [2] 9.0 [3] PostgreSQL [4] SQLite 10.5 [5] DB2 LUW 11gR1 12cR2 Oracle SQL Server [6] [0] group_concat [1] array_to_string [0] group_concat [1] array_to_string [2] [2] array_agg array_agg [3] string_agg [3] [4] group_concat w/o ORDER BY [5] No ON OVERFLOW clause [6] string_agg announced for vnext

113 New in SQL:2016 JSON LISTAGG ROW PATTERN MATCHING DATE FORMAT POLYMORPHIC TABLE FUNCTIONS

114 Modern SQL has evolved beyond the relational idea.

115 Modern SQL has evolved beyond the relational idea. If you are using SQL like 25 years ago, you are doing it wrong!

116 Modern SQL has evolved beyond the relational idea. If you are using SQL like 25 years ago, you are doing it wrong! A lot has happened since SQL-92.

117

118 I have shown you a few features today

119 I have shown you a few features today There are hundreds more to discover

120 @ModernSQL modern-sql.com My other website: Training & co: