Advanced and Complex SQL Coding

Transcription

1 Platform: iseries Application Development Advanced and Complex SQL Coding Sheryl M. Larsen Sheryl M. Larsen, Inc. May 12, 2011 Gateway 400 User Group, St. Louis, MO 1

2 Sheryl M. Larsen, Inc. DB2 SQL Consulting & Education Sheryl Larsen is an internationally recognized researcher, consultant and lecturer, specializing in DB2 and is known for her extensive expertise in SQL. She co-authored a book, DB2 Answers, Osborne- McGraw-Hill, She was voted an inaugural IBM Information Champion 2009, IDUG Speaker Hall of Fame and, a member of IBM s DB2 Gold Consultants program since Sheryl has 25 years experience in DB2, has published articles, white papers, webtv, animated access paths: Google Sheryl Bmc.com Ca.com Ibm.com Currently, she is President of the Midwest Database Users Group (mwdug.org), and owns Sheryl M. Larsen, Inc., a firm specializing in Advanced DB2 Consulting and Education. Contact: smlsql@comcast.net (630) Sheryl M. Larsen, Inc

3 Table of Contents Joins Subqueries Outer Joins Common Table Expressions Materialized Query Tables EXCEPT INTERCEPT Browsing Result Sets Sheryl M. Larsen, Inc

4 Name: SQL Skill Self Assessment Before: Level Assessment = YOU CAN FULLY UNDERSTAND THE FEATURE AND PROPER USE OF: 0 You think SQL is a new energy drink 1 Simple SELECT statements, WITH clause, ORDER BY After: WHERE clauses, BETWEEN, LIKE, IN(list), =, >=, >, <, <=, <>, NOT IN(list), NOT LIKE, NOT BETWEEN, DISTINCT Table joins (inner, outer, full), UNION, UNION ALL, CONCAT, static CURSORs, FOR UPDATE OF, ROW_NUMBER, EXCEPT, INTERCEPT noncorrelated and correlated subqueries, EXISTS, NOT EXISTS, FETCH FIRST x ROWS ONLY, OPTIMIZE FOR x ROWS, MERGE, TRUNCATE Indexable, Stage1 and Stage 2 predicate evaluation, multirow FETCH/INSERT, GET DIAGNOSTICS, Scalar full SELECT Table expressions/ common table expressions, GROUP BY, HAVING, IS NOT DISTINCT FROM, embedded dynamic SQL, ORDER BY/FETCH FIRST in subselect, FETCH CONTINUE, EXCEPT/INTERCEPT CASE expressions, Global Temporary Table (GTT), Declared Temporary Table (DTT), Dynamic Scrollable cursors, SEQUENCES/IDENTITY Queries involving > 10 tables, INSERT within (SELECT, UPDATE, DELETE, MERGE), Star Schema, GROUP BY expression, Browsing result sets MQT (Materialized Query Tables), Recursive SQL, UNION in Views, > 30 useful Built-in Functions, DENSE_RANK, RANK Codes effective and efficient SQL applying performance rules and knows when to use each SQL feature appropriately

5 Full SELECT 225 Tables in SQL Statement Limit 225 directly or indirectly identifying base table references: SELECT.. FROM T1, T2,.T200 WHERE C1 IN (SELECT.. FROM T300 more tables WHERE ) UNION ALL (SELECT FROM T400 more tables WHERE) ORDER BY FETCH FIRST 22 ROWS ONLY WITH UR; Sheryl M. Larsen, Inc Subquery Subselect 5

6 Inner Join Produce a list of the accounts numbers, current servicing agent start date for all active Insurance Payment Plan accounts. SELECT FROM WHERE AND AND AND T42.PLCY_ACCT_NUM,T52.SERV_ASGN_START_DT CLIENT T42,AGENT T52 T42.PLCY_ID = T52.PLCY_ID T42.LOB = S T42.CLNT_PLCY_END_DT IS NULL T52.SERV_ASGN_END_DT IS NULL List Most Restrictive Condition first within predicate type =, IN BETWEEN, >=, >,<,<= LIKE, Noncorrelated Subqueries Correlated Subqueries Sheryl M. Larsen, Inc

7 Self Join Needed for Intra-Table Column Comparisons» WHERE A.COL2 = A.COL5» On different rows (accounts, policies, items, orders, other unique ids, etc.)» What Students Have the Same Classes? Point to Different Rows SELECT SSS.SID, SSS.CLASS_ID, SSS.TIME_ID FROM SMLU_STUDENT_SCHED SS,SMLU_STUDENT_SCHED SSS WHERE SSS.CLASS_ID = SS.CLASS_ID AND SSS.TIME_ID = SS.TIME_ID AND SSS.SID <> SS.SID AND SSS.SID NOT IN(list challenged student IDs) AND SS.SID = :challenged-sid Sheryl M. Larsen, Inc Self Join to get same class/time rows Homework Challenged 7

8 Subqueries over Joins Needed for Intra-Table Column Comparisons» What Students Have at least one of the same Classes as a given student? SELECT SS.SID FROM SMLU_STUDENT_SCHED SS WHERE SS.SID NOT IN(list challenged student IDs) AND EXISTS (SELECT TRUE OR FALSE FROM SMLU_STUDENT_SCHED SSS WHERE SSS.CLASS_ID = SS.CLASS_ID AND SSS.TIME_ID = SS.TIME_ID AND SSS.SID <> SS.SID AND SSS.SID = :challenged-sid) Homework Challenged Sheryl M. Larsen, Inc

9 Left/Right Outer Joins Get all of Client ids for single males» Regardless if there are any addresses SELECT FROM ON WHERE AND T40.CLNT_ID, COALESCE(T71.EML_ADDR, 'NONE AVAIL') CLIENT T40 LEFT OUTER JOIN T71 T40.CLNT_ID = T71.CLNT_ID T40.MRTL_STTS = 'S' T40.GNDR = 'M' Client T40 T71 Sheryl M. Larsen, Inc

10 Inner Join versus Left Join SELECT Columns FROM TABX X LEFT JOIN TABY Y ON X.COL1 = Y.COL1 TABX Exceptions TABY Exceptions Inner Join SELECT Columns FROM TABX X, TABY Y WHERE X.COL1 = Y.COL1 Sheryl M. Larsen, Inc

11 Nested Common Table Expressions WITH COMMON1 (column list) AS (SELECT PART, SUBPART, 1 FROM LEGO WHERE PART = L1 ), COMMON2 (column list) AS SELECT L.PART, L.SUBPART, N+1 FROM COMMON1 T1, LEGO L WHERE T1.SUBPART = L.PART AND N < 10 ), COMMON3 (column list) AS SELECT L.PART, L.SUBPART, N+1 FROM COMMON2 T1, BUILDING_BLOCKS L WHERE T1.SUBPART = L.PART AND N < 10 ) SELECT * FROM COMMON1, TODDLERS2 WHERE..; Sheryl M. Larsen, Inc

12 Materialized Query Tables Automatic Summary Tables (ASTs) are a subset of MQTs Can be ANY valid SELECT statement CREATE TABLE MQT1 AS SELECT dozens of aggregates seven table join nine column GROUP BY Or use your imagination.. ; Sheryl M. Larsen, Inc

13 MQTs Use them as multi-table indexes with or without aggregates Optimizer is aware of all MQTs (for dynamically bound only for z/os!) or direct reference for static Auto query rewrite of base table queries to use MQTs if:» CURRENT REFRESH AGE = ANY» CURRENT MAINTINED TABLE TYPES = SYSTEM, USER, ALL» ENABLE QUERY OPTIMIZATION is used» Referential constraints between base tables exists Sheryl M. Larsen, Inc Matters more than you think 13

14 . To Build or not to Build Generic MQT strategies»huge MQT with many indexes»many small MQTs with few indexes Optimal MQT strategy»most popular queries as MQTs Common tables Common predicate sets Common data translations»indexes on the popular ones to allow access to subset of MQT Sheryl M. Larsen, Inc

15 Popular Table Pivoting Syntax SELECT SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END) AS JAN_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END) AS FEB_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END) AS MARCH_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END) AS APRIL_AMT Generated at Execution, SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END) AS MAY_AMT FROM SALES WHERE CUST_ID = :hv JAN_AMT FEB_AMT MARCH_AMT APRIL_AMT MAY_AMT 238, , , , Sheryl M. Larsen, Inc

16 MQT INDEX: CUST_ID MQT CREATE TABLE CUST_PIVOT AS ( SELECT CUST_ID, SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END) AS JAN_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END) AS FEB_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END) AS MARCH_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END) AS APRIL_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END) AS MAY_AMT FROM SALES GROUP BY CUST_ID) DATA INITIALLY DEFERRED REFRESH DEFERRED MAINTAINED BY SYSTEM ENABLE QUERY OPTMIZATION; Sheryl M. Larsen, Inc

17 Query Speed Up SELECT CUST_ID, SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END) AS JAN_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END)AS FEB_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END)AS MARCH_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END)AS APRIL_AMT, SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END)AS MAY_AMT FROM SALES WHERE CUST_ID = 12 Accessed SELECT * FROM CUST_PIVOT WHERE CUST_ID = 12 CUST_PIVOT CUST_ID JAN_AMT FEB_AMT MARCH_AMT APRIL_AMT MAY_AMT , , , , , , , , , , ,038, , , , Sheryl M. Larsen, Inc at Execution 17

18 EXCEPT, INTERSECT R1 R2 NAME Baker Baker NAME Charlie SELECT AS R1 UNION SELECT.. AS R2 SELECT AS R1 UNION ALL SELECT.. AS R2 SELECT AS R1 EXCEPT SELECT.. AS R2 SELECT AS R1 EXCEPT ALL SELECT.. AS R2 SELECT AS R1 INTERSECT SELECT.. AS R2 SELECT AS R1 INTERSECT ALL SELECT.. AS R2 Charlie NAME NAME NAME NAME NAME NAME Baker Baker Charlie Charlie Baker Charlie Baker Baker Charlie Charlie Baker Sheryl M. Larsen, Inc Baker Baker 18

19 Find Things that Have ALL List the clients that have at least one active policy in ALL lines of business offered by Insurance Company. SELECT DISTINCT PR1.CLNT_ID FROM POLICY_ROLE PR1 All All WHERE NOT EXISTS (SELECT DISTINCT P1. LOB FROM POLICY P1 ) EXCEPT (SELECT DISTINCT PR2.LOB FROM POLICY_ROLE PR2 WHERE PR2.CLNT_ID = PR1.CLNT_ID AND PR2.CLNT_PLCY_END_DT IS NULL ) A F H L A F H L - = False A F H L - = H True A F L Sheryl M. Larsen, Inc

20 EXCEPT Example If the employee works on every project located in Denver, then list the employee s social security number and name. SELECT NAME, SSN FROM EMPLOYEE E WHERE NOT EXISTS ((SELECT PROJECT.PNUMBER FROM PROJECT WHERE PLOCATION = DENVER') EXCEPT All P11 P26 P84 P99 (SELECT W.PNUMBER FROM WORKSON WHERE E.SSN = W.SSN)); Test one SSN at a time If their list matches They win! Sheryl M. Larsen, Inc

21 Platform: z/os Application Development Repositioning Cursors Sheryl M. Larsen Sheryl M. Larsen, Inc. 21

22 EMP Table and Cursor EMPNO EMP_NAME EMP_ADDR EMP_PHONE 1 JOE DOWNERS GROVE SAM ELMHURST JIM AURORA KELLY LIBERTYVILLE MIKE BARRINGTON DELCLARE EMPCSR CURSOR FOR SELECT EMP_NAME, EMP_ADDR, EMP_PHONE FROM EMP WHERE EMPNO > :lastempno ORDER BY EMPNO FETCH FIRST 2 ROWS ONLY; Sheryl M. Larsen, Inc

23 Get First 2 Rows EMPNO EMP_NAME EMP_ADDR EMP_PHONE 1 JOE DOWNERS GROVE SAM ELMHURST JIM AURORA KELLY LIBERTYVILLE MIKE BARRINGTON SET :lastempno = 0, :page0 in COM AREA OPEN EMPCSR; FETCH EMPCSR FOR 2 ROWS; Store :lastempno = :last-fetched-empno in COM AREA and :page1 Display screen Sheryl M. Larsen, Inc

24 Get Next 2 Rows EMPNO EMP_NAME EMP_ADDR EMP_PHONE 1 JOE DOWNERS GROVE SAM ELMHURST JIM AURORA KELLY LIBERTYVILLE MIKE BARRINGTON Get :lastempno from COM AREA (=2 for page1) OPEN EMPCSR; FETCH EMPCSR FOR 2 ROWS; Store :lastempno = :last-fetched-empno in COM AREA and :page2 Sheryl M. Larsen, Inc

25 Go Back 1 Page EMPNO EMP_NAME EMP_ADDR EMP_PHONE 1 JOE DOWNERS GROVE SAM ELMHURST JIM AURORA KELLY LIBERTYVILLE MIKE BARRINGTON Get :lastempno from COM AREA for Page 0 (=0) OPEN EMPCSR; FETCH EMPCSR; Store :firstempno and :page2 in COM AREA FETCH EMPCSR; Store :lastempno = :last-fetched-empno in COM AREA Always go an extra page back so the > works Sheryl M. Larsen, Inc

26 BRWSUM Table Makes row unique, should have index COL1.COL2.COL3.COL4 COL1 COL2 COL3 COL4 COL5 A 1 TT 99 Data. A 1 UU 77 Data. A 4 SS 66 Data. B 2 RR 66 Data. B 3 RR 77 Data. B 3 RR 88 Data. B 3 SS 66 Data. B 4 SS 99 Data. B 4 UU 88 Data. C 1 SS 66 Data. C 1 SS 77 Data. Sheryl M. Larsen, Inc

27 Declare Cursor DELCLARE BRWSUM1 CURSOR FOR SELECT COL5 FROM BRWSUM WHERE? AND? AND? AND? AND? AND? AND? ORDER BY COL1, COL2, COL3, COL4 FETCH FIRST 5 ROWS ONLY; COL1 COL2 COL3 COL4 COL5 A 1 TT 99 Data. A 1 UU 77 Data. A 4 SS 66 Data. B 2 RR 66 Data. What should these be? B 3 RR 77 Data. B 3 RR 88 Data. B 3 SS 66 Data. B 4 SS 99 Data. B 4 UU 88 Data. C 1 SS 66 Data. C 1 SS 77 Data. Sheryl M. Larsen, Inc

28 Declare BRWSUM1Cursor DELCLARE BRWSUM1 CURSOR FOR SELECT COL5 FROM BRWSUM WHERE ((COL1 = :col1-last AND COL2 = :col2-last AND COL3 = :col3-last AND COL4 > :col4-last) OR (COL1 = :col1-last AND COL2 = :col2-last AND COL3 > :col3-last) OR (COL1 = :col1-last AND COL2 > :col2-last) OR (COL1 > :col1-last)) ORDER BY COL1, COL2, COL3, COL4 FETCH FIRST 5 ROWS ONLY; Sheryl M. Larsen, Inc COL1 COL2 COL3 COL4 COL5 A 1 TT 99 Data. A 1 UU 77 Data. A 4 SS 66 Data. B 2 RR 66 Data. B 3 RR 77 Data. B 3 RR 88 Data. B 3 SS 66 Data. B 4 SS 99 Data. B 4 UU 88 Data. C 1 SS 66 Data. C 1 SS 77 Data. 28

29 Get First 5 Rows SET :col1-last = low values, :col2-last = 0, :col3-last = low values,:col4-last = 0, :page0 in COM AREA OPEN BRWSUM1; FETCH BRWSUM1 FOR 5 ROWS; Store :col1-4-last = :last-fetchedrow in COM AREA and :page1 Display screen COL1 COL2 COL3 COL4 COL5 A 1 TT 99 Data. A 1 UU 77 Data. A 4 SS 66 Data. B 2 RR 66 Data. B 3 RR 77 Data. B 3 RR 88 Data. B 3 SS 66 Data. B 4 SS 99 Data. B 4 UU 88 Data. C 1 SS 66 Data. C 1 SS 77 Data. 29 Sheryl M. Larsen, Inc

30 Get Next 5 Rows 30 Get from COM AREA :col1-last = B, :col2-last = 3, :col3-last = RR,:col4-last =77 OPEN BRWSUM1; ( (COL1 = B AND COL2 = 3 AND COL3 = RR AND COL4 > 77) OR (COL1 = B AND COL2 = 3 AND COL3 > RR ) OR (COL1 = B AND COL2 > 3) OR (COL1 > B )) First Screen FETCH BRWSUM1 FOR 5 ROWS; Store :col1-4-last = :last-fetched-row in COM AREA and :page2 Display screen Sheryl M. Larsen, Inc COL1 COL2 COL3 COL4 COL5 A 1 TT 99 Data. A 1 UU 77 Data. A 4 SS 66 Data. B 2 RR 66 Data. B 3 RR 77 Data. B 3 RR 88 Data. B 3 SS 66 Data. B 4 SS 99 Data. B 4 UU 88 Data. C 1 SS 66 Data. C 1 SS 77 Data.

31 Go Back 1 Page Get from COM AREA for page0 :col1-last = low values :col2-last = 0, :col3-last = low values,:col4-last =0 31 OPEN BRWSUM1; ( (COL1 = low AND COL2 = 0 AND COL3 = low AND COL4 > 0) OR (COL1 = low AND COL2 = 0 AND COL3 > low) OR (COL1 = low AND COL2 > 0) OR (COL1 > 0)) FETCH BRWSUM1; Loop 5 times Last Screen Store :col1-4-last = :last-fetched-row in COM AREA and :page1 Display screen Sheryl M. Larsen, Inc COL1 COL2 COL3 COL4 COL5 A 1 TT 99 Data. A 1 UU 77 Data. A 4 SS 66 Data. B 2 RR 66 Data. B 3 RR 77 Data. B 3 RR 88 Data. B 3 SS 66 Data. B 4 SS 99 Data. B 4 UU 88 Data. C 1 SS 66 Data. C 1 SS 77 Data.

32 Platform: z/os Application Development Thank You! 32 Sheryl M. Larsen Sheryl M. Larsen, Inc. iseries Information Center, Version 5 Release 3