Set Review 3 SQL Working with Sets Question: number of customers in China? Count what s here Locations Customers Question: who are the customers in Ch

Transcription

1 Advanced SQL Set Processing Rob Bestgen IBM DB2 for i SQL is a very powerful language for access data Utilizing it means leveraging database to do more of the work for you Accessing a table is only part of the story Goal: turn data into information 2 1

2 Set Review 3 SQL Working with Sets Question: number of customers in China? Count what s here Locations Customers Question: who are the customers in China? Return what s here 2

3 Sets A set must be well defined For any given object, it must be unambiguous whether or not the object is an element of the set defined The elements of a set may be described without actually being listed There is no particular order of elements in a set 5 Record Level Access versus SQL Record Level Access You tell the database what to do AND You tell the database how to do it SQL You tell the database what to do BUT You do not tell the database how to do it And that s ok! 3

4 Tell DB2 Combine Your SQL Multiple SQL Statements DECLARE CURSOR cursor1 FOR SELECT col1, col2,... col9 FROM t1 WHERE cust_id = 1234 AND transaction_date = ' '; OPEN cursor1; SINGLE SQL Request INSERT INTO t2 SELECT col1, col2,... col9 FROM t1 WHERE cust_id = 1234 AND transaction_date = ' '; DO FETCH cursor1 INTO :c1, :c2,..., :c9; INSERT INTO t2 VALUES( :c1, :c2,..., :c9 ); UNTIL ( no more data ); CLOSE cursor1; Tell DB2 Combine Your SQL Multiple SQL Statements DECLARE CURSOR cursor1 FOR SELECT custid FROM order_table WHERE ord_date = '2012/11/03'; OPEN cursor1; DO FETCH cursor1 INTO :v_custid; SELECT cust_name, cust_address INTO :v_name, :v_address FROM cust_table WHERE custid= :v_custid; /* Process customer data */ UNTIL ( no more data ); CLOSE cursor1; SIMPLIFIED SQL Request DECLARE CURSOR cursor1 FOR SELECT c.cust_name, c.cust_address FROM order_table o, cust_table c WHERE ord_date = '2012/11/03 AND o.custid = c.custid; OPEN cursor1; DO FETCH cursor1 INTO :v_name, v_address; /* Process customer data */ UNTIL ( no more data ); CLOSE cursor1; 4

5 SQL 1 SQL 2 SQL 3 Query QTEMP Query QTEMP Query 9 Query Query Query Single SQL Statement 10 5

6 SQL SELECT Sequence The virtual sequence of the (hypothetical) operations is: 1. FROM clause 2. WHERE clause 3. GROUP BY clause 4. HAVING clause 5. ORDER BY clause 6. SELECT clause 7. FETCH FIRST clause 11 SQL DML Think in SETs INSERT (a set) UPDATE (a set) DELETE (a set) SELECT (a set) 12 6

7 SQL SELECT The fullselect (aka final select) is a component of a: Select statement ALTER TABLE statement for the definition of a MQT CREATE TABLE statement CREATE VIEW statement DECLARE GLOBAL TEMPORARY TABLE statement INSERT statement SET transition-variable statement SET VARIABLE statement UPDATE statement DELETE statement Assignment statement 13 SQL SELECT The subselect is a component of a fullselect a set within a set A subselect defines a result table derived from the tables or views identified in its FROM clause a result set A scalar-subselect is a subselect, enclosed in parentheses, that returns a single result row and a single result column If the result of the subselect is no rows, then the null value is returned An error is returned if there is more than one row in the result A set with at most one element A subselect can show up just about anywhere in an SQL statement 14 7

8 SQL subselect possibilities SELECT (scalar subselect) Construct and use sets within sets, or sets from other sets A query within a query FROM (sub select) WHERE (and/or JOIN) (subselect or scalar subselect) GROUP BY (scalar subselect) HAVING (subselect or scalar subselect) ORDER BY (subselect or scalar subselect) FETCH FIRST 15 VIEWs Common Table Expressions Derived Tables (nested table expression) SubQueries 16 8

9 SQL Query Anatomy Statement complexity is a function of: The data model The data The amount of work to be accomplished in one request The desired or required result set to be returned Max statement length: 2MB Max number of tables referenced: 1, VIEWs Common Table Expressions Derived Tables (nested table expression) SubQueries 9

10 SQL Views CREATE VIEW activeemployees AS SELECT * FROM Employees WHERE start_date <= CURRENT_DATE AND ( end_date IS NULL OR end_date >= CURRENT_DATE ); SELECT * FROM activeemployees WHERE state = 'NV' ORDER BY lastname ; SQL Common Table Expressions WITH activeemployees AS ( SELECT * FROM Employees WHERE start_date <= CURRENT_DATE AND ( end_date IS NULL OR end_date >= CURRENT_DATE ) ) SELECT * FROM activeemployees WHERE state = 'NV' ORDER BY lastname ; 10

11 SQL Derived Table Expressions (aka Nested Table Expressions) SELECT * FROM ( SELECT * FROM Employees WHERE start_date <= CURRENT_DATE AND ( end_date IS NULL OR end_date >= CURRENT_DATE ) ) t1 WHERE state = 'NV' ORDER BY lastname ; CTEs Thinking in Sets What if you want the top 10 list of customers based on total sales? SELECT customer_name, SUM(sales) FROM SALES GROUP BY customer_name ORDER BY SUM(sales) DESC FETCH FIRST 10 ROWS ONLY CTE approach pretty similar, but WITH top10 (customer_name, total_sales) AS (SELECT customer_name, SUM(sales) FROM SALES GROUP BY customer_name ORDER BY SUM(sales) DESC FETCH FIRST 10 ROWS ONLY) SELECT * from top10 11

12 CTEs Thinking in Sets What if you want the list of customers who were in the top 10 for two consecutive years? Think in sets WITH top10_2013 (customer_name, total_sales) AS (SELECT customer_name, SUM(sales) FROM SALES WHERE YEAR=2013 GROUP BY customer_name ORDER BY SUM(sales) DESC FETCH FIRST 10 ROWS ONLY), top10_2014 (customer_name, total_sales) AS (SELECT customer_name, SUM(sales) FROM SALES WHERE YEAR=2014 GROUP BY customer_name ORDER BY SUM(sales) DESC FETCH FIRST 10 ROWS ONLY) SELECT Y1.customer_name, Y1.total_sales AS sales2013 Y2.total_sales AS sales2014 FROM top10_2013 Y1 INNER JOIN top10_2014 Y2 ON Y1.customer_name = Y2.customer_name CTEs Logical Step-by-Step and Work Tables CTE provides the same step-by-step approach without the overhead of populating physical tables WITH t1 AS (SELECT shipdate, customer, phone, orderkey, linenumber FROM item_fact i INNER JOIN cust_dim c ON c.custkey = i.custkey WHERE discount=0.08), t2 AS (SELECT customer, phone, orderkey, linenumber, year, quarter FROM t1 INNER JOIN time_dim t ON t.datekey = shipdate) SELECT * FROM t2; 12

13 CTEs - Legacy Data Model Support Useful when dealing with legacy data models Different record types in a single table Multiple columns stored in a single column WITH hold_mail(hold_address,hold_date) AS ( SELECT mail_address, chg_date FROM mail_changes WHERE chg_type = HOLD ), start_mail(start_address,start_date) AS ( SELECT mail_address, chg_date FROM mail_changes WHERE chg_type = 'START ) SELECT hold_address, hold_date, start_date FROM hold_mail INNER JOIN start_mail ON hold_address=start_address Comparison: Views, CTEs, and NTEs View CTE NTE Permanent object that must be managed YES NO NO Can be reused within a query Can be reused by other queries and jobs YES YES NO YES NO NO Supports recursion YES YES, NO Definition can reference host variables NO, reference to view can use host variables YES YES Definition can contain ORDER BY NO, reference to view can be ordered YES* YES* * ORDER BY only honored if used in conjunction with FETCH FIRST clause 13

14 Business Problem Produce a report of quantity and revenue for year 2014 select o.year, o.month, sum(o.quantity), sum(o.revenue) from orders o where o.year = 2014 group byo.year, o.month order by o.year, o.month; Summary by Year, Quarter, Month , , , , , , , , , , , , Business Problem Squared Summary by Year, Quarter, Month Summary by Year, Quarter, Month , , ,000 99,990 Big Table Many Rows Detailed Data , , , , , , , , , , , , , , , ,982 Big Table Many Rows Detailed Data , , , ,963 Year Quarter Month Quantity Delta Quantity Revenue Delta Revenue , , , , , , ,900 +1, , , , , , , , , ,100 +1, , ,

15 Summarize 2014 and LEFT OUTER JOIN to 2010 Summary , , , , , , , , , , , ,949 Drill Across ,000 99, , , , , , , , , , ,963 Summary for 2014 by Quarter, Month Summary for 2010 by Quarter, Month 29 Summarize 2014 and INNER JOIN to 2010 Summary , , , , , , , , , , , ,949 Drill Across ,000 99, , , , , , , , , , ,963 Summary for 2014 by Quarter, Month Summary for 2010 by Quarter, Month 30 15

16 Derived Tables Year Quarter Month Quantity Delta Current - 1 Revenue Delta Current - 1 select t1.year, t1.quarter, t1.month, t1.quantity, (t1.quantity - t2.quantity) as quantity_delta, t1.revenue, (t1.revenue - t2.revenue) as revenue_delta from (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue from orders o where o.year = 2014 group by o.year, o.month) t , , , , , , ,900 +1, , , , , , , , , ,100 +1, , , , , , , , , , , , , , , left outer join (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue from orders o where o.year = 2010 group by o.year, o.month) t2 on order by t1.quarter = t2.quarter and t1.month = t2.month t1.year, t1.quarter, t1.month; ,000 99, , , , , , , , , , ,963 Common Table Expressions Option 1 with t1 as (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue from orders o 32 where o.year = 2014 group by o.year, o.month), t2 as (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue select from orders o where o.year = 2010 group by o.year, o.month) t1.year, t1.quarter, t1.month, t1.quantity, (t1.quantity - t2.quantity) as quantity_delta, t1.revenue, (t1.revenue - t2.revenue) as revenue_delta from t1 left outer join t2 on t1.quarter = t2.quarter and t1.month = t2.month order by t1.year, t1.quarter, t1.month; , , , , , , , , , , , , ,000 99, , , , , , , , , , ,963 Year Quarter Month Quantity Delta Current - 1 Revenue Delta Current , , , , , , ,900 +1, , , , , , , , , ,100 +1, , ,986 16

17 Common Table Expressions Option 2 with sums as (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue select from group by orders o o.year, o.month), t1.year, t1.quarter, t1.month, t1.quantity, (t1.quantity - t2.quantity) as quantity_delta, t1.revenue, (t1.revenue - t2.revenue) as revenue_delta from sums t1 left outer join sums t2 on (t1.quarter = t2.quarter and t1.month = t2.month and t2.year = 2010) where t1.year = 2014 order by t1.year, t1.quarter, t1.month; ,000 99, , , , , , , , , , , , , , , , , , , , , Views Option 1 create view orders2014 as (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue from orders o where year = 2014 group by o.year, o.month); , , , , , , , , , , , ,949 create view orders2010 as (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue from orders o where year = 2010 group by o.year, o.month); ,000 99, , , , , , , , , , ,963 17

18 Views Option 1 select t1.year, t1.quarter, t1.month, t1.quantity, (t1.quantity - t2.quantity) as quantity_delta, t1.revenue, (t1.revenue - t2.revenue) as revenue_delta from orders2014 t1 left outer join orders2010 t2 on t1.quarter = t2.quarter and order by t1.month = t2.month t1.year, t1.quarter, t1.month; Year Quarter Month Quantity Delta Current - 1 Revenue Delta Current , , , , , , ,900 +1, , , , , , , , , ,100 +1, , , Views Option create view orders_sum_yqm as (select o.year, o.month, sum(o.quantity) as quantity, sum(o.revenue) as revenue from orders o group by o.year, o.month); ,000 99, , , , , , , , , , , , , , , , , , , , , , ,949 18

19 Views Option 2 select t1.year, t1.quarter, t1.month, t1.quantity, (t1.quantity - t2.quantity) as quantity_delta, t1.revenue, (t1.revenue - t2.revenue) as revenue_delta from orders_sum_yqm t1 left outer join orders_sum_yqm t2 on (t1.quarter = t2.quarter and t1.month = t2.month and t2.year = 2010) where t1.year = 2014 order by t1.year, t1.quarter, t1.month; Year Quarter Month Quantity Delta Quantity Revenue Delta Revenue , , , , , , ,900 +1, , , , , , , , , ,100 +1, , , Business Problem Revisited Produce a report of quantity and revenue for years, by quarter, with year total select o.year, sum(o.quantity), sum(o.revenue) from orders o where o.year > 2010 group by grouping sets (o.year, o.quarter), (o.year) order by o.year desc, o.quarter; Summary by Year, Quarter Year Quarter Quantity Revenue , , , , , , , , ,750 1,074, , ,

20 Another Business Problem - SubQuery Produce a revenue report for only customers in Minnesota select o.year, o.month, sum(o.revenue) from orders o where o.year = 2011 and o.customer_number in ( select c.customer_number from customers c group by o.year, o.month order by o.year, o.month; where c.state = Minnesota ) Orders Customers MN 39 Another Business Problem Inner Join Produce a revenue report for only customers in Minnesota 40 select o.year, o.month, sum(o.revenue) from orders o inner join customers c on o.customer_number = c.customer_number where o.year = 2011 and c.state = Minnesota group by o.year, o.month order by o.year, o.month; Orders Customers MN What must be true about customers.customer_number? 20

21 Yet Another Business Problem CTE, Inner Join and SubQuery Produce a revenue report for only customers in Minnesota with revenue less than their previous year s total 41 with previous_year as (select oo.customer_number, sum(oo.revenue) as total_revenue select group by oo.customer_number) o.customer_number, 2011 sum(o.revenue) Orders from orders oo where oo.year = 2010 Customers MN from orders o inner join customers c 2010 on o.customer_number = c.customer_number Orders where o.year = 2011 and c.state = Minnesota group by o.customer_number having sum(o.revenue) < (select py.total_revenue from previous_year py where o.customer_number = py.customer_number) order by o.customer_number; Dating Problems 42 21

22 Business Problem Revisited (Derived Columns) orders shipdate quantity revenue year(shipdate) quarter(shipdate) month(shipdate) 43 Business Problem Revisited (Derived Columns) Produce a report of quantity and revenue for year based on ship date select year(o.shipdate) as year, quarter(o.shipdate) as quarter, month(o.shipdate) as month, sum(o.quantity), sum(o.revenue) from orders o where year(o.shipdate) = 2011 group byyear(o.shipdate), quarter(o.shipdate), month(o.shipdate) order by year(o.shipdate), quarter(o.shipdate), month(o.shipdate); Note: 3 years of data 44 Summary by Year, Quarter, Month , , , , , , , , , , , ,949 22

23 Business Problem Revisited (No Date Columns) Create and use a date master table orders shipdate quantity revenue 45 Date Master table to Simplify Time Based Queries Dates and their Attributes Orders INNER JOIN? Note: 3 years of data 46 23

24 Business Problem Revisited (Date Master) Produce report of quantity and revenue for year 2011 select d.year, d.quarter, d.month, sum(o.quantity), sum(o.revenue) from orders o inner join dates d on o.shipdate = d.datekey where d.year = and anything else related to dates group by d.year, d.quarter, d.month order by d.year, d.quarter, d.month; Note: 3 years of data 47 Summary by Year, Quarter, Month , , , , , , , , , , , ,949 Views ORDERS + DATES aka hiding the date master orders shipdate = datekey dates 48 create view orders_plus_dates as ( select d.*, o.orderdate, o.shipdate, o.quantity, o.revenue from orders o, inner join dates d on o.shipdate = d.datekey); 24

25 Business Problem Solved (Date Master) Produce report of quantity and revenue for year 2011 select year, quarter, month, sum(quantity), sum(revenue) from orders_plus_dates where year = and anything else related to dates group byyear, quarter, month order by year, quarter, month; Note: 3 years of data 49 Summary by Year, Quarter, Month , , , , , , , , , , , ,

26 VIEWs Common Table Expressions Derived Tables (nested table expression) SubQueries 51 Need help using the newest DB2 for i technologies? Are you getting the most out DB2 for i? IBM DB2 for i Consulting and Services Database & Query modernization DB2 WebQuery Database design, features and functions DB2 SQL performance analysis and tuning Data warehousing and Business Intelligence DB2 for i education and training Contact: Mike Cain mcain@us.ibm.com IBM Systems and Technology Group Rochester, MN USA 26

27 IBM DB2 for i Course Offerings

28 Special notices This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area. Information in this document concerning non-ibm products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied. All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions. IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice. IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies. All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generallyavailable systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment. Special notices (cont.) IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/os, i5/os (logo), IBM Business Partner (logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pseries, Rational, RISC System/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xseries, z/os, zseries, AIX 5L, Chiphopper, Chipkill, Cloudscape, DB2 Universal Database, DS4000, DS6000, DS8000, EnergyScale, Enterprise Workload Manager, General Purpose File System,, GPFS, HACMP, HACMP/6000, HASM, IBM Systems Director Active Energy Manager, iseries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, System i, System p, System p5, System Storage, System z, Tivoli Enterprise, TME 10, Workload Partitions Manager and X-Architecture are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org. UNIX is a registered trademark of The Open Group in the United States, other countries or both. Linux is a registered trademark of Linus Torvalds in the United States, other countries or both. Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both. Intel, Itanium, Pentium are registered trademarks and Xeon is a trademark of Intel Corporation or its subsidiaries in the United States, other countries or both. AMD Opteron is a trademark of Advanced Micro Devices, Inc. Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries or both. TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC). SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC). NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both. AltiVec is a trademark of Freescale Semiconductor, Inc. Cell Broadband Engine is a trademark of Sony Computer Entertainment Inc. InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Other company, product and service names may be trademarks or service marks of others. 28