SQL Saturday Cork Welcome to Cork. Andrea Martorana Tusa T-SQL advanced: Grouping and Windowing

Transcription

1 SQL Saturday Cork Welcome to Cork Andrea Martorana Tusa T-SQL advanced: Grouping and Windowing

2 Andrea Martorana Tusa T-SQL Advanced: Grouping and Windowing

3 Speaker info First name: Andrea. Last name: Martorana Tusa. Italian, working by Widex a danish company which manufactures hearing aids, as BI Specialist. Previously worked for 15 years as BI developer in an italian bank. Focused on database development, datawarehousing, cube development, reporting, data analysis, etc. Speaker at SQL Saturdays, and other community-driven events in Europe, (MS Cloud Summit, SQL Konferenz, SQL Nexus, SQL Days, Dataminds Connect...). Speaker in webinars for PASS Italian VC, DW/BI VC. Author for sqlservercentral.com, sqlshack.com, UGISS (User Group Italiano SQL Server).

4 Agenda Grouping - Aggregate functions CHECKSUM_AGG COUNT_BIG GROUPING Windows functions OVER() clause Ranking, Aggregate and Analytics functions

5 Ranking functions ROW_NUMBER RANK DENSE_RANK NTILE

6 Grouping

7 Aggregate functions AVG CHECKSUM_AGG COUNT COUNT_BIG GROUPING GROUPING_ID MAX MIN STDEV STDEVP SUM VAR VARP

8 Aggregate functions GROUPING It s a function to be used in a SELECT, HAVING and ORDER BY statement. It indicates whether a specified column expression in a GROUP BY list is aggregated or not. 1 for aggregated columns 0 for not aggregated colums Very useful for mapping aggregate data in a resultset.

9 Aggregate functions GROUPING_ID It s a function to be used in SELECT, HAVING, or ORDER BY statement along with a GROUP BY. Basically it computes the level of grouping. It s powerful and versatile and can help in mapping various levels of aggregation in a resultset, increasing its readability.

10 Grouping GROUPY BY A clause of a SELECT statement used to cluster and aggregate data SELECT YEAR(OrderDate) AS OrderYear, MONTH(OrderDate) AS OrderMonth, COUNT(OrderID) AS TotalOrders FROM Sales.Orders GROUP BY YEAR(OrderDate), Month(OrderDate) ORDER BY OrderYear, OrderMonth So far nothing new. But...

11 Grouping... what if you want to create a subtotal or grandtotal in GROUPY BY? There are some operators that can help, adding more fetaures to the statement. ROLLUP CUBE GROUPING SETS

12 Grouping GROUP BY ROLLUP Creates a group for each combination of column expressions. The results are computed into subtotals and grand totals. SELECT Country, Region, SUM(Sales) AS TotalSales FROM Sales.Orders GROUP BY Country, Region WITH ROLLUP; Country Region TotalSales Country NULL TotalSales NULL NULL TotalSales

13 Grouping GROUP BY CUBE Creates groups for all possible combinations of columns. For the GROUP BY CUBE operator you get the following outcome: SELECT Country, Region, SUM(Sales) AS TotalSales FROM Sales GROUP BY Country, Region WITH CUBE; Country Region TotalSales NULL Region TotalSales Country NULL TotalSales NULL NULL TotalSales

14 Grouping GROUP BY GROUPING SETS GROUPING SETS is a powerful estensions of the clause. It gives you the ability to combine multiple aggregations into one single GROUP BY clause. The results can be considered the equivalent of multiple UNION ALL, one for each of the specified groups.

15 Demo GROUP BY: ROLLAP CUBE GROUPING SETS GROUPING GROUPING_ID

16 Windowing

17 Windowing A window is a subset of rows defined by a common descriptor inside a dataset, to whom it can be applied a function called window function to return a single value for each row of the open frame. The strenght of the partitioning and window functions concepts lay in the flexibility they can offer for calculation at row level, even inside aggregations.

18 Windowing Yearly Total Yearly Total Global Total Avg Sale Window 1 Window 2 Window 3

19 Windowing The basic syntax for defining a window frame relies on few key statements: OVER ( [ <PARTITION BY clause> ] [ <ORDER BY clause> ] [ <ROW or RANGE clause> ] )

20 OVER the key for «opening» windows OVER is the core clause to the foundations of the partitioning logic. Inside a SQL statement OVER() sets up the window, i.e. a subset of rows within a query result set, based on userdefined partitioning and ordering criteria. Once defined the rows interval, the associated window function can be applied and a value is computed for each row in the window.

21 OVER the key for «opening» windows The clause OVER has been used since SQL Server 2005 in ranking functions. Starting from SQL Server 2012, has been expanded to include aggregate and analytic functions. Ranking: RANK, DENSE_RANK, Aggregate: AVG, MAX, MIN, SUM, Analytics: CUME_DIST, LAG, LEAST, FIRST_VALUE, LAST_VALUE,

22 ROWS / RANGE how open is your window? The clauses ROWS and RANGE let you specify the exact subset of rows of the windows frame to which apply the windowing function. Schematically, with the OVER clause you define a sliding window; with ROWS and RANGE you decide how open your window is. [ROWS RANGE] BETWEEN <Start expr> AND <End expr>

23 ROWS / RANGE [ROWS RANGE] BETWEEN <Start expr> AND <End expr> <Start expr>: UNBOUNDED PRECEDING: The window starts in the first row of the partition <offset> PRECEDING or FOLLOWING CURRENT ROW: The window starts in the current row <End expr>: UNBOUNDED FOLLOWING: The window ends in the last row of the partition <offset> PRECEDING or FOLLOWING CURRENT ROW: The window ends in the current row

24 ROWS / RANGE [ROWS RANGE] BETWEEN <Start expr> AND <End expr> Note that: ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW can be abbreviated as ROWS UNBOUNDED PRECEDING If you don t indicate a window frame clause, the default is RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, that might lead to unexpected outcomes

25 ROWS vs RANGE The ROWS clause limits the rows within a partition by specifying a fixed number of rows preceding or following the current row. The RANGE clause includes all the rows within the partition that have the same ORDER BY values as the current row. Usually it is said that with ROWS you define the windows frame on a physical level, with RANGE on a logical level and this has impact on performances.

26 ROWS operation... PARTITION BY CustomerName ORDER BY Year, Month UNBOUNDED PRECEDING 6 PRECEDING CURRENT ROW 4 FOLLOWING UNBOUNDED FOLLOWING

27 RANGE operation... ORDER BY CustomerCategoryName, TranYear UNBOUNDED PRECEDING CURRENT ROW UNBOUNDED FOLLOWING

28 Demo OVER() ROWS RANGE

29 Window analytic function Analytic functions compute an aggregate value based on a group of rows. However, unlike aggregate functions, they can return multiple rows for each group. You can use analytic functions to compute moving averages, running totals, percentages or top-n results within a group. BOL:

30 Windows analytic functions Offset functions CUME_DIST FIRST_VALUE LAG LAST_VALUE LEAD PERCENTILE_CONT PERCENTILE_DISC PERCENT_RANK

31 LAG / LEAD The LAG/LEAD functions return a column from a previous/following row in the partition, with respect to the current row. The row from which obtain the value is indicated by the specified offset LAG/LEAD support windows partition and ordering clause, but don t support window frame. The syntax for these functions is: LAG LEAD (scalar_expression [,offset] [,default]) OVER ( [ partition_by_clause ] order_by_clause )

32 FIRST_VALUE / LAST_VALUE The FIRST_VALUE / LAST_VALUE functions return the first or the last value into an ordered frame. Partitioning, ordering and framing are supported. The syntax for these functions is: FIRST_VALUE LAST_VALUE ( [scalar_expression ] ) OVER ( [ partition_by_clause ] order_by_clause [ rows_range_clause ] )

33 Demo LAG LEAD FIRST_VALUE LAST_VALUE

34 Questions?

35 Grazie!