Performance Issue : More than 30 sec to load. Design OK, No complex calculation. 7 tables joined, 500+ millions rows

Transcription

1

2 Bienvenue

3 Nicolas

4 Performance Issue : More than 30 sec to load Design OK, No complex calculation 7 tables joined, 500+ millions rows Denormalize, Materialized Views, Columnstore Index Less than 5 sec to load

5 # T C 1 8 The truth is in the stars with data architectures for Tableau Nicolas Lerose Sales Consultant Tableau

6 Agenda Understand Tableau VizQL to Abstract to SQL Leverage the Data Access Layer Set up database for performance in Tableau Use Hyper to speed up slow data sources

7 What is Tableau?

8 It s a Visual Dynamic SQL Generator That s the magic Self-service visual data analysis VizQL Statement A sentence like statement describing pills on shelves and cards along with aggregations in the Tableau Interface Abstract Query An XML statement describing the query needed for a Viz including select, aggregation, calculations and filter Physical Query The native syntax SQL or MDX sent to the database engine to execute Database Data source using SQL or MDX Interface over ODBC

9 Tableau Interface to SQL SELECT GROUP BY WHERE CALCULATIONS 1-Simple 2-Table 3-LOD 4-Passthrough As Above Local to Tableau, Not in SQL Nested SQL Nested SQL

10 Demo

11 Is Data Architecture the Magic Bullet? No! Workbook Design Calculations Queries Data Env.

12 Data Server

13 The Data Server for Performance Shared Meta-data & Client Drivers and Web Authoring a must Caution with performance overhead (this will get better in the future) Data Server Communication Process complicates things

14 Remember This? That s the magic Self-service visual data analysis VizQL Statement A sentence like statement describing pills on shelves and cards along with aggregations in the Tableau Interface Abstract Query An XML statement describing the query needed for a Viz including select, aggregation, calculations and filter Physical Query The native syntax SQL or MDX sent to the database engine to execute Database Data source using SQL or MDX Interface over ODBC

15 Become This A more verbose XML statement called a SQL Proxy describing the query needed for a Viz including select, aggregation, calculations, groups and filters SQL Proxy Statement Tableau Data Server Tableau Server Process converts SQL Proxy Statements into Physical SQL Self-service visual data analysis VizQL Statement A sentence like statement describing pills on shelves and cards along with aggregations in the Tableau Interface Abstract Query An XML statement describing the query needed for a Viz including select, aggregation, calculations and filter Physical Query The native syntax SQL or MDX sent to the database engine to execute Database Data source using SQL or MDX Interface over ODBC

16 Database Views

17 Database Views SQL defined in the view DDL is transparent to Tableau Tableau treat database views like tables and generate SQL against them Views can be used to set access control, row and column level security Views can be used to prepare semantic and business rules Aggregated views can be materialized to speed up performance

18 Object Model

19 Object Model Data Source From organized tables to a single large table (denormalized) Object Model Preserve normalized tables

20 Indexing

21 Indexing A copy of selected columns of data from a table Used to quickly locate data without having to search every row in the table Includes an address or direct link to the complete row of data it was copied from Create indexes on joining/filtering dimensions

22 Join culling

23 Join culling Configure referential integrity in your database (if you can) Assume Referential Integrity (if it makes sense) This option impacts only inner joins and does not affect data sources with a single table Dimension filters always take advantage of Join Culling (single table queried) lineitem table orders table join result table OrderKey OrderKey OrderKey (Foreign Key) Product Sales Amount Discount (Primary Key) Sales Clerk 1 10 Inch Tablet 600 5% 1 Adam Hart 1 10 Inch Case 50 5% 2 Denny Joy 2 Smart Phone 300 2% 3 Emily Burns 3 Desk Lamp 50 10% 1 Power Bank 25 0% (Join Key) Product Sales Amount Discount Sales Clerk 1 10 Inch Tablet 600 5% Adam Hart 1 10 Inch Case 50 5% Adam Hart 2 Smart Phone 300 2% Denny Joy 3 Desk Lamp 50 10% Emily Burns 1 Power Bank 25 0% Adam Hart SELECT SUM([Sales Amount]) FROM [lineitem] Answer: 1025 SELECT SUM([Sales Amount]) FROM [lineitem] INNER JOIN [orders] ON [lineitem].[orderkey] = [orders].[orderkey] Answer = 1025

24 Partition pruning

25 Partition pruning Table is split up into two or more tables Combined with data source filters on partitioning column to avoid scanning the full table Ex: X billions Rows, but really only looked at last week this year vs previous in most analytics

26 Column stores

27 Column stores Stores data tables by column rather than by row Well-suited for OLAP-like workloads which typically involve highly complex queries over all data Higher compression Scan only the column(s) needed!

28 Our Star Cust. Segment Cust. Segment ID Cust. Segment Name Supplier Supplier ID Supplier Name. Market Market ID Market Name Orders Cust. Segment ID Product ID Sales Tax Product Product ID Product Name Employee Emp. ID Emp. Name

29 Demo

30 Hyper

31 Denormalized, Flat Objects, NoSQL? Simple : Use Hyper! Use for JSON, Statistical Files, CSV, TXT, XLS(X), ACCDB, WDC and other File Based Data Source Use for Generic ODBC, Custom SQL & NoSQL Connections Use for Overworked data Warehouses Use for Persistence or Joins, Federated Joins, Unions & Calculations

32 Why is Hyper so Fast? Memory optimized CPU optimized Query compilation Query optimizations

33 Normalized Extracts

34 Normalized Extracts (2018.3) Extracts can now store multiple tables separately Great for avoiding cartesian product when joining with user table for row level security

35 Federated Joins

36 How Cross Database Joins Work 1. Query each connection for row-level data 2. Bring row-level data to Tableau 3. Join row-level data in Tableau

37 But data has mass* X-DB Joins move (a lot of) data around Reduce data movement to the minimum Use Extracts when possible * and therefore it has gravity & inertia.

38 Alternative Federation Engine Feature Goal: Reduce data movement to the minimum

39 Demo

40 Conclusion

41 Conclusion Consider the Design, Calculations and Queries before the Data Architecture Use indexes and partitions to speed up filters and joins Use persistence and Materialized Views to answer common problems Use Hyper or Columnstores for Dashboard or Document Acceleration Object Model will change the way Tableau generate queries

42 We re looking for Object Models alpha testers Do you have many data sources with blends in one workbook? Do you use a LoD expressions to correct aggregations? Do you use scaffolding tables to ensure that data is not lost through joins?

43 R E L AT E D S E S S I O N S Turbocharging Tableau with Hyper Wed, Oct 24 3:30pm 4:30pm MCCNO-L2-240 Thu, Oct 25 2:15pm 3:15pm MCCNO-L2-240 Hot, hotter, Hyper How to handle big data Tue, Oct 23 4:00pm 5:00pm MCCNO-L2-240 Thu, Oct 25 10:45am 11:45am MCCNO-L2-240

44 Please complete the session survey from the My Evaluations menu in your TC18 app

45 Merci!

46