Performance Tuning for the BI Professional. Jonathan Stewart

Size: px

Start display at page:

Download "Performance Tuning for the BI Professional. Jonathan Stewart"

Amanda Brooks
6 years ago
Views:

1 Performance Tuning for the BI Professional Jonathan Stewart

3 Jonathan Stewart Business Intelligence Consultant SQLLocks,

5 Agenda Shared Solutions SSIS SSRS SSAS Power BI Summary

6 Performance tuning starts in the architecture The physician can bury his mistakes, but the architect can only advise his client to plant vines. - Frank Lloyd Wright

7 Shared Solutions

8 Shared Solutions Take only data needed Sort in the database! Use procedures when available Index, Index, Index and Stats Aggregate any data as needed Uncorrelated sub queries instate of correlated sub queries Convert data in query, not in SSIS/SSRS/SSAS/etc Consider Maintenance Network Package size

9 SSIS

10 SSIS Design choices Think outside the box Understand performance differences

11 SSIS - Buffer performance Synchronous Vs. Asynchronous Synchronous VS Asynchronous Streaming Row-Based Partially Blocking Blocking Audit Export Column Data Mining Query Aggregate Cache Transform Import Column Merge Fuzzy Grouping Character Map Lookup (no or partial cache) Merge Join Fuzzy lookup Conditional Split OLE DB Command Pivot Row Sampling Copy Column Script Component Term Lookup Sort * Data Conversion Slowly Changing Dimension Unpivot Term Extraction Derived Column Union All Script Component (w/ async output) Lookup (full cache) Multicast Percent Sampling Row Count Script Component (with synch output)

12 SSIS - Buffer performance Synchronous Vs. Asynchronous Streaming transforms manipulate data in place and are the fastest

13 SSIS - Buffer performance Synchronous Vs. Asynchronous Synchronous VS Asynchronous Streaming Row-Based Partially Blocking Blocking Audit Export Column Data Mining Query Aggregate Cache Transform Import Column Merge Fuzzy Grouping Character Map Lookup (no or partial cache) Merge Join Fuzzy lookup Conditional Split OLE DB Command Pivot Row Sampling Copy Column Script Component Term Lookup Sort * Data Conversion Slowly Changing Dimension Unpivot Term Extraction Derived Column Union All Script Component (w/ async output) Lookup (full cache) Multicast Percent Sampling Row Count Script Component (with synch output)

14 SSIS - Buffer performance Synchronous Vs. Asynchronous Streaming transforms manipulate data in place and are the fastest Row-Based, exactly what it sounds like Slower as a result (like a cursor)

15 SSIS - Buffer performance Synchronous Vs. Asynchronous Synchronous VS Asynchronous Streaming Row-Based Partially Blocking Blocking Audit Export Column Data Mining Query Aggregate Cache Transform Import Column Merge Fuzzy Grouping Character Map Lookup (no or partial cache) Merge Join Fuzzy lookup Conditional Split OLE DB Command Pivot Row Sampling Copy Column Script Component Term Lookup Sort * Data Conversion Slowly Changing Dimension Unpivot Term Extraction Derived Column Union All Script Component (w/ async output) Lookup (full cache) Multicast Percent Sampling Row Count Script Component (with synch output)

16 SSIS - Buffer performance Synchronous Vs. Asynchronous Streaming transforms manipulate data in place and are the fastest Row-Based, exactly what it sounds like Slower as a result (like a cursor) Partially blocking have multiple inputs, single output Generally can be done in the database (faster) After the transformation writes a row to the new buffer, the row is available to the next transformation even while the partially blocking transformation continues processing remaining rows.

17 SSIS - Buffer performance Synchronous Vs. Asynchronous Synchronous VS Asynchronous Streaming Row-Based Partially Blocking Blocking Audit Export Column Data Mining Query Aggregate Cache Transform Import Column Merge Fuzzy Grouping Character Map Lookup (no or partial cache) Merge Join Fuzzy lookup Conditional Split OLE DB Command Pivot Row Sampling Copy Column Script Component Term Lookup Sort * Data Conversion Slowly Changing Dimension Unpivot Term Extraction Derived Column Union All Script Component (w/ async output) Lookup (full cache) Multicast Percent Sampling Row Count Script Component (with synch output)

18 SSIS - Buffer performance Synchronous Vs. Asynchronous Streaming transforms manipulate data in place and are the fastest Row-Based, exactly what it sounds like Slower as a result (like a cursor) Partially blocking have multiple inputs, single output Generally can be done in the database (faster) After the transformation writes a row to the new buffer, the row is available to the next transformation even while the partially blocking transformation continues processing remaining rows. Blocking transforms also multiple input, single output If possible use the database instead Reads all rows in the pipeline before writing rows to the new buffer, which prevents downstream components from starting (hence blocking)

19 SSIS - Buffer performance Synchronous Vs. Asynchronous Synchronous VS Asynchronous Streaming Row-Based Partially Blocking Blocking Audit Export Column Data Mining Query Aggregate Cache Transform Import Column Merge Fuzzy Grouping Character Map Lookup (no or partial cache) Merge Join Fuzzy lookup Conditional Split OLE DB Command Pivot Row Sampling Copy Column Script Component Term Lookup Sort * Data Conversion Slowly Changing Dimension Unpivot Term Extraction Derived Column Union All Script Component (w/ async output) Lookup (full cache) Multicast Percent Sampling Row Count Script Component (with synch output)

20 SSIS - Resolutions Use SQL Command instead of "Table or View" option (unless fast load which uses Bulk Insert) Table or View options uses openrowset Will load rows one at a time

21 SSIS - Resolutions

22 SSIS - Resolutions

23 SSIS - Resolutions Use SQL Command instead of "Table or View" option (unless fast load which uses Bulk Insert) Table or View options uses openrowset Will load rows one at a time When loading data uncheck check constraint if possible

24 SSIS - Resolutions

25 SSIS - Resolutions Use SQL Command instead of "Table or View" option (unless fast load which uses Bulk Insert) Table or View options uses openrowset Will load rows one at a time When loading data uncheck check constraint if possible Drop Indexes on staging tables then re-add after load Partition destination table if loading from multiple sources Either use multiple tables then combine or partitions, allows table lock

26 SSIS - Resolutions

27 SSIS - Resolutions Use SQL Command instead of "Table or View" option (unless fast load which uses Bulk Insert) Table or View options uses openrowset Will load rows one at a time When loading data uncheck check constraint if possible Drop Indexes on staging tables then re-add after load Partition destination table if loading from multiple sources Either use multiple tables then combine or partitions, allows table lock Tune buffers Enable BufferSizeTuning logging event Monitor log to see how many rows were put in buffer Adjust DefaultBufferMaxRows and DefaultBufferSize as needed to get optimal rows in buffer

28 SSIS - Resolutions

29 SSIS - Resolutions Use SQL Command instead of "Table or View" option (unless fast load which uses Bulk Insert) Table or View options uses openrowset Will load rows one at a time When loading data uncheck check constraint if possible Drop Indexes on staging tables then re-add after load Partition destination table if loading from multiple sources Either use multiple tables then combine or partitions, allows table lock Tune buffers Enable BufferSizeTuning logging event Monitor log to see how many rows were put in buffer Adjust DefaultBufferMaxRows and DefaultBufferSize as needed to get optimal rows in buffer Configure Rows per Batch and Maximum Insert Commit Size in OLEDB destination.

30 SSIS - Resolutions

31 SSIS - Summary Think speed when building your packages! Use the database! Understand SSIS is all about buffer management! Try to avoid asynchronous transforms!

32 SSRS

33 SSRS - Datasets Don t create unnecessary datasets Delete any unused, every dataset will be executed You can use one dataset for multiple report parts (i.e. Tablix, Matrix and Chart) If parameter has fixed values, specify them and avoid a parameter dataset (.i.e. months). Tune them in the database! If using a similar dataset for multiple reports, consider using one and filters

34 SSRS Calculated Fields Avoid complex calculated fields as much as possible Better to create and do the work at the query level Less load on reporting server Eases maintenance

35 SSRS Page Layouts Avoid anything that references the entire report will slow down rendering Total Page Numbers Report will have to fully render to show value Will give appearance of slow report Avoid complex expressions requiring pagination Report processor can assume any complex expression in the page header or footed might contain a direct or indirect reference to [&TotalPages] Avoid interactive sorts when possible Avoid blank pages Increases rendering time Use page breaks

36 SSRS Page Layouts (cont d) For reports that have many instances of text boxes, set CanGrow and CanShrink on text boxes to FALSE. By default, each cell in a Tablix data region contains a text box, so that total number of text boxes that must be rendered can grow quickly

37 SSRS Page Layouts (cont d) For reports that have many images, set AutoSize on images to a different value such as Fit. Avoid horizontal page breaks when they are not required. Review the margins, column widths, and white space in a report. Set the KeepTogether property on Tablix members only when you must control the specific rendering behavior for a Tablix data region. The KeepTogether feature requires extra processing when page breaks are calculated.

38 SSRS - Summary Datasets! Calculated Fields Strategic Page Layouts

39 SSAS/Power BI The Vertipaq Engine

40 SSAS/Power BI Tabular is aggregate aware and will summarize all numeric columns by default. Change them on design. Explicitly create calculations as measures using DAX. Filter data in table import before even processing Eliminate unnecessary columns to make the model smaller Take only vital and non-vital columns Create views in the source to easier manage the data Estimate Current Size and Growth Carefully There are tools like BISM Memory Report and Vertipaq Analyzer that can further help in this process.

41 SSAS/Power BI Tabular is forgiving of bad data model design Best practice is star schema Include only needed columns! Cardinality Optimize DAX Bad DAX effect worse as model grows Test DAX in DAX Studio Performance Tuning of Tabular models in SSAS 2012 whitepaper. Monitor User patterns and Train Users

42 SSAS - Paul Turley s Performance Logger

43 SSAS - Paul Turley s Performance Logger

44 SSAS/Power BI Narrow long tables are faster than short and wide Remove unused fields Integers are faster than strings Be aware of your cardinality! Do you really need a live connection? Be cautious of slicer use! Cross filters multiply effect! Check memory usage! Kasper de Jonge s Power Pivot Memory Usage tool in Excel.

45 SSAS/Power BI Filter statement runs against every row. Slow measures and calculations can cause problems Test your DAX in DAX Studio. Consider processing in the database. Measures are calculated during query execution Slow measures and calculations can cause problems Don t make your DAX overly complex

46 SSAS\Power BI DAX Functions

47 SSAS\Power BI DAX Functions

48 SSAS\Power BI DAX Functions

49 SSAS\Power BI - Summary Design with high performance in mind Estimate your model Take what you need Train your users Power BI is fast! Remember Cardinality! Use the DB if needed! Check your DAX! Think Now!

50 Summary Use the database!! Make sure we understand synchronous vs asynchronous transforms Understand the performance implications of our other SSIS architectural choices Be as explicit as possible in SSRS Understand what increases rendering time Take caution and be explicit in our tabular models Design with capacity in mind Think through your Power BI design Remember, speed isn t just in the actual performance, consider maintenance and troubleshooting speed too! Finally the user

51 QUESTIONS

53 Jonathan Stewart Business Intelligence

Deccansoft Software Services. SSIS Syllabus

Deccansoft Software Services. SSIS Syllabus Overview: SQL Server Integration Services (SSIS) is a component of Microsoft SQL Server database software which can be used to perform a broad range of data migration, data integration and Data Consolidation