More SQL in MySQL 8.0

Transcription

1

2 More SQL in MySQL 8.0 Norvald H. Ryeng Sofware Development Senior Manager MySQL Optmizer Team November, 08

3 Program Agenda 3 4 Common table expressions (CTEs) Window functons JSON_TABLE Demo

4 Theory 4

5 Common Table Expressions (CTEs) Alternatve to derived tables or views SELECT FROM (subquery) AS derived, t CREATE VIEW derived AS (subquery); SELECT FROM derived, t WITH derived AS (subquery) SELECT FROM derived, t Can refer to other CTEs WITH cte AS (subquery), cte AS (SELECT FROM cte ) SELECT FROM cte Can refer to itself, creatng a recursive CTE 5

6 Recursive CTEs WITH RECURSIVE cte AS (SELECT FROM table_name UNION [DISTINCT ALL] SELECT FROM cte, table_name) SELECT FROM cte; /* "seed" SELECT */ /* "recursive" SELECT */ A recursive CTE refers to itself in a subquery The seed SELECT is executed once to create the inital data subset The recursive SELECT is executed repeatedly Stops when iteraton doesn t generate any new rows Set cte_max_recursion_depth to limit recursion (default 000) Useful to traverse hierarchies (parent/child, part/subpart) 6

7 Recursive CTE Example CREATE TABLE employee (number INTEGER PRIMARY KEY, name VARCHAR(00), manager INTEGER REFERENCES employee (number)); INSERT INTO employee VALUES (, 'Alice', NULL), (, 'Bob', ), (3, 'Bety', ), (4, 'Charlie', ), (5, 'Cherise', ), (6, 'Chandler', ), (7, 'Chris', 3), (8, 'Camilla', 3), (9, 'Dave', 8), (0, 'Denise', 8); WITH RECURSIVE emps AS ( SELECT *, name AS reportng_chain FROM employee WHERE manager IS NULL UNION ALL SELECT e.number, e.name, e.manager, CONCAT(e.name, ', ', m.reportng_chain) FROM employee AS e JOIN emps AS m ON e.manager = m.number ) SELECT * FROM emps; number name manager Alice NULL Bob Bob, Alice 3 Bety Bety, Alice 4 Charlie Charlie, Bob, Alice 5 Cherise Cherise, Bob, Alice 6 Chandler Chandler, Bob, Alice 7 Chris 3 Chris, Bety, Alice 8 Camilla 3 Camilla, Bety, Alice 9 Dave 8 Dave, Camilla, Bety, Alice 0 Denise 8 Denise, Camilla, Bety, Alice reportng_chain Alice 7

8 Recursive CTE Example CREATE TABLE employee (number INTEGER PRIMARY KEY, name VARCHAR(00), manager INTEGER REFERENCES employee (number)); INSERT INTO employee VALUES (, 'Alice', NULL), (, 'Bob', ), (3, 'Bety', ), (4, 'Charlie', ), (5, 'Cherise', ), (6, 'Chandler', ), (7, 'Chris', 3), (8, 'Camilla', 3), (9, 'Dave', 8), (0, 'Denise', 8); WITH RECURSIVE emps AS ( SELECT *, name AS reportng_chain FROM employee WHERE manager IS NULL UNION ALL SELECT e.number, e.name, e.manager, CONCAT(e.name, ', ', m.reportng_chain) FROM employee AS e JOIN emps AS m ON e.manager = m.number ) SELECT * FROM emps; number name manager Alice NULL Bob Bob, Alice 3 Bety Bety, Alice 4 Charlie Charlie, Bob, Alice 5 Cherise Cherise, Bob, Alice 6 Chandler Chandler, Bob, Alice 7 Chris 3 Chris, Bety, Alice 8 Camilla 3 Camilla, Bety, Alice 9 Dave 8 Dave, Camilla, Bety, Alice 0 Denise 8 Denise, Camilla, Bety, Alice reportng_chain Alice 8

9 Window Functons Similar to aggregaton functons Computes one value based on multple rows But does not group Aggregaton functon Window functon 9

10 Window Functon Example Sum up total salary per department SELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id) AS dept_total FROM employee ORDER BY dept_id, name; The OVER keyword signals a window functon PARTITION disjoint set of rows in result set name Newt Dag Ed Fred Jon Michael Newt Lebedev Pete Jef Will dept_id NULL salary NULL dept_total

11 Window Functon Example Sum up total salary per department SELECT name, dept_id, salary, SUM(salary) OVER (PARTITION BY dept_id) AS dept_total FROM employee ORDER BY dept_id, name; The OVER keyword signals a window functon PARTITION disjoint set of rows in result set name Newt Dag Ed Fred Jon Michael Newt Lebedev Pete Jef Will dept_id NULL salary NULL dept_total

12 JSON_TABLE INSERT INTO t (json_col) VALUES ( '{ "people": [ { "name":"john Smith", "address":"780 Mission St, San Francisco, CA 9403"}, { "name":"sally Brown", "address":"75 37th Ave S, St Cloud, MN 9403"}, { "name":"john Johnson", "address":"6 Roosevelt Trail, Raymond, ME 0407"} ] }' ); SELECT people.* FROM t, JSON_TABLE(json_col, '$.people[*]' COLUMNS ( name VARCHAR(40) PATH '$.name', address VARCHAR(00) PATH '$.address')) AS people; name John Smith Sally Brown John Johnson address 780 Mission St, San Francisco, CA th Ave S, St Cloud, MN Roosevelt Trail, Raymond, ME 0407

13 JSON_TABLE Nested Arrays [ { "father":"john", "mother":"mary", "marriage_date":" ", "children": [ { "name":"eric", "age":0 }, { "name":"beth", "age": } ] }, { "father":"paul", "mother":"laura", "children": [ { "name":"sarah", "age":9}, { "name":"noah", "age":3}, { "name":"peter", "age":0} ] } id father married child_id child age John Eric 0 John Beth Paul 0 Sarah 9 Paul 0 Noah 3 Paul 0 3 Peter 0 ] 3

14 JSON_TABLE Nested Arrays JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', married INTEGER EXISTS PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) )) id father married child_id child age John Eric 0 John Beth Paul 0 Sarah 9 Paul 0 Noah 3 Paul 0 3 Peter 0 4

15 Playtme 5

16 Demo Warning! The following queries are intended to demonstrate the power of the features This is not necessarily the best way of doing things This is not necessarily the easiest way of doing things This is not necessarily smart but it's fun! :-) 6

17 JSON_TABLE 7

18 Birth Order # First child in a family =, second child =, etc. SELECT fam.*, RANK() OVER w AS nth_child FROM families, JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', mother VARCHAR(30) PATH '$.mother', married INTEGER EXISTS PATH '$.marriage_date', marriage_date DATE PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) ) ) AS fam WINDOW w AS (PARTITION BY id ORDER BY age DESC) ORDER BY id, child_id; 8

19 Families as Table # Use JSON_TABLE to convert JSON document to a table representng the same info CREATE TABLE families_tab SELECT fam.* FROM families, JSON_TABLE (families, '$[*]' COLUMNS ( id FOR ORDINALITY, father VARCHAR(30) PATH '$.father', mother VARCHAR(30) PATH '$.mother', marriage_date DATE PATH '$.marriage_date', NESTED PATH '$.children[*]' COLUMNS ( child_id FOR ORDINALITY, child VARCHAR(30) PATH '$.name', age INTEGER PATH '$.age' ) ) ) AS fam; 9

20 Families from Table to JSON # Use JSON_ARRAYAGG to produce JSON SELECT JSON_PRETTY( JSON_OBJECT( 'father',any_value(father), 'mother',any_value(mother), 'marriage_date',any_value(marriage_date), 'children',json_arrayagg(json_object('name',child,'age',age)) ) ) FROM families_tab GROUP BY id; # Getng rid of "marriage_date" : null is lef as an exercise to the reader 0

21 GPS Tracking

22 Timeline CREATE TABLE tmeline ( ts TIMESTAMP, pos POINT SRID 436 ); INSERT INTO tmeline VALUES (' :30:00', ST_SRID(POINT( , ), 436)), (' :30:05', ST_SRID(POINT( , ), 436)), (' :30:0', ST_SRID(POINT( , ), 436)), (' :30:5', ST_SRID(POINT( , ), 436)), (' :30:0', ST_SRID(POINT( , ), 436)), (' :30:5', ST_SRID(POINT( , ), 436)), (' :30:30', ST_SRID(POINT( , ), 436)), (' :30:35', ST_SRID(POINT( , ), 436)), (' :30:40', ST_SRID(POINT( , ), 436)), (' :30:45', ST_SRID(POINT( , ), 436)), (' :30:50', ST_SRID(POINT( , ), 436)), (' :30:55', ST_SRID(POINT( , ), 436)) ; INSERT INTO tmeline VALUES (' :47:00', ST_SRID(POINT( , ), 436)), (' :47:05', ST_SRID(POINT( , ), 436)), (' :47:0', ST_SRID(POINT( , ), 436)), (' :47:5', ST_SRID(POINT( , ), 436)), (' :47:0', ST_SRID(POINT( , ), 436)), (' :47:5', ST_SRID(POINT( , ), 436)), (' :47:30', ST_SRID(POINT( , ), 436)), (' :47:35', ST_SRID(POINT( , ), 436)), (' :47:40', ST_SRID(POINT( , ), 436)), (' :47:45', ST_SRID(POINT( , ), 436)), (' :47:50', ST_SRID(POINT( , ), 436)) ; INSERT INTO tmeline VALUES (' :4:00', ST_SRID(POINT( , ), 436)), (' :4:05', ST_SRID(POINT( , ), 436)), (' :4:0', ST_SRID(POINT( , ), 436)), (' :4:5', ST_SRID(POINT( , ), 436)), (' :4:0', ST_SRID(POINT( , ), 436)), (' :4:5', ST_SRID(POINT( , ), 436)), (' :4:30', ST_SRID(POINT( , ), 436)), (' :4:35', ST_SRID(POINT( , ), 436)), (' :4:40', ST_SRID(POINT( , ), 436)) ;

23 Window Functon to Add Rank # Order by tmestamp. First point =, second point =, etc. CREATE VIEW ranked_tmeline AS SELECT *, RANK() OVER w AS num, TIME_TO_SEC(TIMEDIFF(ts, LAG(ts) OVER w)) AS dif FROM tmeline WINDOW w AS ( PARTITION BY DATE(ts) ORDER BY ts ) ; 3

24 Generate GeoJSON Path # 5 minutes (300 seconds) or more between points -> new trip. WITH RECURSIVE cte AS ( SELECT num AS trip, ts AS start, ts AS stop, num, JSON_ARRAY(JSON_ARRAY(ST_Longitude(pos), ST_Lattude(pos))) AS path FROM ranked_tmeline WHERE dif IS NULL OR dif >= 300 UNION ALL SELECT cte.trip, cte.start, ranked_tmeline.ts, ranked_tmeline.num, JSON_ARRAY_APPEND(cte.path, '$', JSON_ARRAY(ST_Longitude(ranked_tmeline.pos), ST_Lattude(ranked_tmeline.pos))) FROM ranked_tmeline, cte WHERE ranked_tmeline.num - = cte.num AND ranked_tmeline.dif < 300 ) SELECT DISTINCT trip, start, FIRST_VALUE(stop) OVER w AS stop, JSON_OBJECT('type', 'LineString', 'coordinates', FIRST_VALUE(path) OVER w) AS path_json FROM cte WINDOW w AS (PARTITION BY trip ORDER BY num DESC); 4

25 Trips Ploted from GeoJSON 5

26 Feature descriptons and design details directly from the source. htp://mysqlserverteam.com/ 6

27 7

28 Safe Harbor Statement The preceding is intended to outline our general product directon. It is intended for informaton purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functonality, and should not be relied upon in making purchasing decisions. The development, release, and tming of any features or functonality described for Oracle s products remains at the sole discreton of Oracle. 8

29