Data Analysis Expressions (DAX)

Transcription

1 Javier Guillén

2 About Me o Business Intelligence / Data Warehousing Consultant o MCTS/MCITP SQL Server 2008 BI o Microsoft Community Contributor Awardee (2011) o C# & ASP.NET Developer o Worked in Financial Industry for 8 years About Mariner o Microsoft Gold Partner - Business Intelligence o Microsoft Silver Partner Data Platform o Microsoft 2010 US K12 Education Partner of the Year o TDWI 2009 Best Practices Implementation Partner o Microsoft 2008 Worldwide Partner of the Year for Performance Management

3 Session Overview Brief introduction to DAX Define Evaluation Context Compare MDX and DAX rules for establishing context Describe DAX context evaluation and how it can be manipulated Examples using business calculations

4 Data Analysis Expressions (DAX) An expression language used to develop calculations in: PowerPivot SSAS 2011 Business Intelligence Semantic Model Cresent Uses in-memory columnar storage engine (Vertipaq) Can expose itself to clients through relational as well as multidimensional interfaces

5 PowerPivot / DAX advantages as a BI tool Ability to integrate multiple data sources, each containing millions of rows of data Excel-like formula syntax Can reshape in-memory data model to better suit reporting needs Can to publish and automate reports Ability to gain enterprise-wide visibility on spreadsheet popularity and use

6

7 Context Wine cup or two human faces? We interpret reality and assign a context to what we see

8 What is Evaluation Context? In DAX & MDX, it refers the set of rules followed in order to determine how a calculation should be interpreted What makes possible dynamic analysis

9

10 Both languages use a coordinate system to establish how a formula context gets interpreted MDX DAX

11 MDX Example What is the total amount of reseller sales across all products and all years? SELECT FROM WHERE [Measure].[Reseller Sales Amount] ON COLUMNS [Adventure Works] [Product].[Product].[All] $80,450,596.98

12 SELECT [Organization].[Organizations].[Organization Level 01].&[1] ON COLUMNS FROM [Adventure Works] SELECT Head([Employee].[Vacation Hours].Members) ON COLUMNS FROM [Adventure Works] SELECT FROM [Adventure Works] WHERE [Destination Currency].[Destination Currency] SELECT FROM [Adventure Works] $80,450,596.98

13 MDX Rules for Evaluating Context Complete tuples use explicitly stated members. In partial tuples, omitted members are derived using the following sequence: I. Use default member. If absent, then: II. Use [ALL] member. If absent, then: III. Use first member

14 SELECT [Organization].[Organizations].[Organization Level 01].&[1] ON COLUMNS FROM [Adventure Works] DEFAULT MEMBER SELECT Head([Employee].[Vacation Hours].Members) ON COLUMNS FROM [Adventure Works] [ALL] MEMBER SELECT FROM [Adventure Works] WHERE [Destination Currency].[Destination Currency] FIRST MEMBER SELECT FROM [Adventure Works] ALL OF THE ABOVE $80,450,596.98

15 DAX Example What is the total amount or of reseller sales across all products and all years? CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(DimProduct), ALL(DimDate) ) $80,450,596.98

16 CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(DimProduct), ALL(DimDate) ) Column In-memory Vertipaq table

17 SUM(FactResellerSales[SalesAmount]) CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(FactResellerSales[SalesOrderNumber]) ) CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(DimProduct) ) SUMX( FactResellerSales, FactResellerSales[SalesAmount] ) $80,450,596.98

18 DAX Rules for Evaluating Context Intersection of active rows on in-memory tables (including their relationships) DAX makes no assumptions for members omitted in formula Filters can use three types of contexts when deriving what rows qualify as active in a calculation: I. Row Context II. Query Context III. Filter Context

19 SUM(FactResellerSales[SalesAmount]) ONE TABLE NO FILTERS CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(FactResellerSales[SalesOrderNumber]) ) CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(DimProduct) ) SUMX( FactResellerSales, FactResellerSales[SalesAmount] ) ONE TABLE ONE FILTER TWO TABLES ONE FILTER ONE TABLE ROW BY ROW ITERATION $80,450,596.98

20 DAX Row Context Evaluates expression on the current row If the table has a relationship with another table, it can propagate to all columns on that table that are related to the current row

21 In-memory Table in PowerPivot Window

22 DAX Query context Subset of data implicitly retrieved when resolving a formula Based on row and column headers, slicers and report filters Different for each cell on a pivot table

23 Year Bikes Category + Road Bikes + Reseller Sales Pivot Table in Excel

24 DAX Filter Context Added when specifying filter constraints via DAX formula arguments Applies on top of row context and query context

25 CALCULATE( SUM(FactResellerSales[SalesAmount]), DimProduct[Color] = Red ) Year Bikes Category + Road Bikes + Reseller Sales + Red Color Filter Context

26 DAX evaluation context is resolved by: 1. Establishing what in-memory tables and relationships are active in the calculation 2. Finding what rows within those tables are active in the calculation 3. Determining the intersection of active rows for each filter across the different contexts

27 MDX vs DAX: Evaluation Context Comparison Both languages follow precise rules to establish context MDX translate omitted members into actual cube members. DAX makes no translation for omitted members, it only uses members that in the current context

28 MDX can easily achieve the same functionality as DAX s filter and query context Row by row calculations in SSAS are possible using: Measure group expressions Named columns in DSV MDX Crossjoin MDX Crossjoins can have performance degradation with leaf level calculations. The performance impact of using these type of calculations is generally lower in DAX

29

30 DAX functions that affect evaluation context generally take columns or entire tables as input parameters

31 DAX calculations can be: Calculated Columns: Persisted as a new column added to an in-memory table Measures: Output is calculated in the context of all filters applied to a spreadsheet cell

32 Sample Data Model Order Date

33 Row context manipulation: The RELATED Function Returns a related value from another table Requires a relationship between tables

34 RELATED(DimProductSubcategory[EnglishProductSubcategoryName]) DimProduct In-Memory Table

35 A similar function, RELATEDTABLE can be used to summarize member values SUMX( ) RELATEDTABLE(FactResellerSales), FactResellerSales[SalesAmount] DimProduct In-Memory Table

36 Row context manipulation: AggregateX functions Following an iterative process, these functions execute row by row calculations and aggregate all rows at the end AggregateX Functions AverageX CountaX CountX MaxX MinX SumX

37 Signature: AggX(table, expression)

38 Evaluation Context: Sample Case using Row Context SUMX from dimension table, calling on the fact table: The value for each product is different SUMX( ) RELATEDTABLE(FactResellerSales), FactResellerSales[OrderQuantity] * FactResellerSales[UnitPrice] DimProduct In-Memory Table

39 Each product row modifies the context in which the SUMX iteration occurs DimProduct In-Memory Table

40 The new row context creates an inner loop Outer loop Inner loop

41 SUMX from the fact table over the fact table: The value for each product is the total of all products SUMX( ) FactResellerSales, FactResellerSales[OrderQuantity] * FactResellerSales[UnitPrice] FactResellerSales In-Memory Table

42 Each fact row evaluates over the entire fact table. There is no product context to restrict the values FactResellerSales In-Memory Table

43 SUMX as a measure on dimension as well as fact table: The value is exactly the same for both SUMX( ) RELATEDTABLE(FactResellerSales), FactResellerSales[OrderQuantity] * FactResellerSales[UnitPrice] Measure on DimProduct table SUMX( ) FactResellerSales, FactResellerSales[OrderQuantity] * FactResellerSales[UnitPrice] Measure on FactResellerSales table

44 Why? As a calculated column on fact table referencing itself, the context is the entire table As a calculated column on dimension table referencing the fact table, the context is established by following the relationship

45 As a measure, the output is resolved as: Bikes category + current subcategory + Year All Products Category, Subcategory and Year are a function of query context Products are calculated either as: Total for all products in fact table OR Sum of each product totals in dimension table Evaluation context is the same for both

46 Filter context manipulation: Removing / Refining Filters DAX expressions give the ability to control evaluation context I. Use the ALL function to remove filters II. Use the VALUES function to refine filters In addition to the above, the CALCULATE function can further extend the ability to manipulate context

47 ALL Function SUMX( FILTER( ALL(FactResellerSales), RELATED(DimCurrency[CurrencyAlternateKey] ) = "USD"), FactResellerSales[SalesAmount] ) When using a table as a parameter, ALL clears filters in every column of the table All Years + All Categories + All Subcategories + Reseller Sales in USD Currency

48 VALUES Function behaves like the SQL DISTINCT clause COUNTROWS( VALUES(FactResellerSales[SalesOrderNumber]) ) Distinct count of orders by product subcategory

49 CALCULATE Function Arguably the most powerful DAX function Needed because: I. Faster than row by row calculations II. Without it, the ALL and VALUES functions have limited ability to manipulate context

50 Using CALCULATE: Example Year United States + Road Bikes + Reseller Sales Year United States + All Subcategories + Reseller Sales

51 Removing one filter CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(DimSalesTerritory[SalesTerritoryCountry]) ) Year All Countries + Road Bikes + Reseller Sales Year All Countries + All Subcategories + Reseller Sales

52 Removing more than one filter CALCULATE( SUM(FactResellerSales[SalesAmount]), ALL(DimSalesTerritory[SalesTerritoryCountry]), ALL(DimDate[CalendarYear]) ) All Years + All Countries + Road Bikes + Reseller Sales All Years + All Countries + All Subcategories + Reseller Sales

53 Refining one filter CALCULATE( SUM(FactResellerSales[SalesAmount]), FILTER( VALUES(DimDate[MonthNumberOfYear]), DimDate[MonthNumberOfYear] = 1) ) Year Month # 1 + United States + Road Bikes + Reseller Sales Year Month # 1 + United States + All Subcategories + Reseller Sales

54 Refining more than one filter CALCULATE( SUM(FactResellerSales[SalesAmount]), FILTER(VALUES(DimDate[MonthNumberOfYear]), DimDate[MonthNumberOfYear] = 1), FILTER(VALUES(DimProductSubcategory[EnglishProductSubcategoryName]), DimProductSubcategory[EnglishProductSubcategoryName] = "Road Bikes") ) Year Month # 1 + United States + Road Bikes Only + Reseller Sales Year Month # 1 + United States + Road Bikes Only + Reseller Sales

55 Refining Filter Context shortcut The same output value can be achieved with a shorter calculation: CALCULATE( SUM(FactResellerSales[SalesAmount]), DimDate[MonthNumberOfYear] = 1, DimProductSubcategory[EnglishProductSubcategoryName] = "Road Bikes" )

56 If CALCULATE takes care of filters, why do we still need AggX functions? To perform row by row expressions, even inside a CALCULATE formula CALCULATE( SUMX(FactResellerSales, FactResellerSales[OrderQuantity] * FactResellerSales[UnitPrice] ), DimProductSubcategory[EnglishProductSubcategoryName] = "Road Bikes" )

57 CALCULATE Evaluation Context Summary Each filter condition intersects with all other ones to determine formula context CALCULATE( SUM(FactResellerSales[SalesAmount]), DimDate[MonthNumberOfYear] = 1, DimProductSubcategory[EnglishProductSubcategoryName] = "Road Bikes" ) Evaluation context

58 New filters create new context CALCULATE( SUM(FactResellerSales[SalesAmount]), DimDate[MonthNumberOfYear] = 1, DimProductSubcategory[EnglishProductSubcategoryName] = Road Bikes, DimProduct[Color] = Black ) New filter context

59

60 DAX does not require the use of Pivot tables Pivot tables have a number of limitations, for example: Table layout options are limited Adding subtotals for only one row label element cannot be done (is either all or nothing) As a solution, Excel has built-in functions to create free form dashboards using SSAS data: Excel Cube functions

61 What are Excel Cube functions? Part of Excel built-in formula library Allow calling an SSAS tuple or set from a single spreadsheet cell As PowerPivot is really a flavor of SSAS (under the covers), Excel Cube functions: Query DAX measures Use MDX to build sets on DAX calculated columns or PowerPivot tables

62 Two of the most popular Cube functions are CUBEMEMBER and CUBEVALUE Excel Cube Functions CUBEMEMBER CUBEVALUE CUBERANKEDMEMBER CUBEMEMBERPROPERTY CUBESET CUBESETCOUNT CUBEKPIMEMBER Retrieves a specific dimension member Retrieves a cube cellset based on dimension coordinates

63 Modifying Context with Cube functions Suppose we have the following DAX measure: CALCULATE( SUM(FactResellerSales[SalesAmount]), DimProduct[Color] = "Red ) Sales Amount for Red Products Using a CUBEVALUE function we can: Connect to PowerPivot data Specify Query Filters Retrieve the output in a single spreadsheet cell

64 Query context filters DAX measure result

65

66 Weighted Average IF ( ) COUNTROWS(VALUES(DimProductCategory[EnglishProductCategoryName])) = 1, BLANK(), SUMX( FactResellerSales, FactResellerSales[SalesAmount] * FactResellerSales[Order Quantity] ) / SUM(FactResellerSales[OrderQuantity]) Display value only on total row Expression evaluated for each table row Expression evaluated for entire table

67 Rank IF( COUNTROWS(VALUES(FactResellerSales[SalesAmount] ) ) > 0, COUNTROWS( FILTER( ALL( DimSalesTerritory[SalesTerritoryCountry] ), CALCULATE( SUM( FactResellerSales[SalesAmount] ) ) > SUM(FactResellerSales[SalesAmount]) ) ) + 1, BLANK() ) Do not compute on countries with no fact data Count rows with higher values than the current row Filter table which rows will be counted Compare current row value with all other values

68 Summary DAX offers a powerful way to manipulate context in order to create business calculations Calculated Columns and Measures can use DAX to manipulate the context of a calculation Evaluation Context is established through the joint operation of three context types: Query context Row Context Filter Context

69 Thank you! Javier Guillén LinkedIn: Blog: