Martin Cairney. The Why and How of Partitioned Tables

Transcription

1 Martin Cairney The Why and How of ed Tables

2 Housekeeping Mobile Phones please set to stun during the session

3 Session Evaluation

4 Martin Cairney Microsoft Data Platform MVP Microsoft Certified Trainer Organiser SQL Saturday Melbourne

5 Agenda

6 Agenda How is data stored? Why do we want to use it? How do we configure and manage it? Performance and Pitfalls

7 How is data stored?

8 Allocation Units Table / Index Storage In_Row Data LOB Data Row_Overflow Data Each page type is an Allocation Unit (sys.allocation_units) The assignment of pages of each type to a Table is tracked in the IAM pages

9 Hobbits HOBTS Heap Or B-Trees Table without a Clustered Index = HEAP Each Index = B-Tree Each HOBT has at least an In-Row Allocation Unit, and may have the others too (For non-columnstore indexes) No Hobbits were harmed in the making of this slide

10 s Our table and indexes can also be split into a series of ranges on Enterprise Edition (all editions from SQL Server 2016 SP1) Each range can be stored on a different FileGroup, but this is not a requirement Crucially each range is a distinct HOBT So we can have many HOBTS per table

11 How Do They Fit Together? TABLE Index... CLUSTERED INDEX INDEX1 INDEX X HOBT C-1 HOBT C-Y HOBT 1-1 HOBT 1-Y... HOBT X Allocation Unit IN_ROW LOB ROW_OVERFLOW

12 How is data stored? Demo

13 Why Do We Want To Use ing?

14 Benefits Of ing You can transfer or access subsets of data quickly and efficiently, while maintaining the integrity of a data collection. For example, an operation such as loading data from an OLTP to an OLAP system takes only seconds, instead of the minutes and hours the operation takes when the data is not partitioned. Achieved through Switching Normally a meta-data only operation (see the internal structures from previous demos) Requires the same structure and constraints ranges in the non-partitioned table being switched

15 Benefits Of ing You can perform maintenance operations on one or more partitions more quickly. The operations are more efficient because they target only these data subsets, instead of the whole table. For example, you can choose to compress data in one or more partitions or rebuild one or more partitions of an index. A partitioned index can be managed at the partition level In 2016, TRUNCATE can be issued at the partition level Compression can be set at the partition level Incremental statistics stats at the partition level which are merged back into the main histogram If partitions are on separate filegroups, then these can be set read-only, backed up separately, DBCC checks separately

16 Benefits Of ing You may improve query performance, based on the types of queries you frequently run and on your hardware configuration. For example, the query optimizer can process equi-join queries between two or more partitioned tables faster when the partitioning columns in the tables are the same, because the partitions themselves can be joined. You can improve performance by enabling lock escalation at the partition level instead of a whole table. This can reduce lock contention on the table. When the partition key is in the predicate, then partition elimination can occur Tables with the same partition function can benefit from collocated joins effectively each partition joins with the corresponding one, minimising resources for each join

17 Configure & Manage ing

18 Design Function CREATE PARTITION FUNCTION mypf_function (<parameter_type>) AS RANGE [ LEFT RIGHT ] FOR VALUES ( val1, val2, val3,...,valx ) ; <parameter_type> : will match the data type of the column in the table used to partition the table Function Name: must be unique in the database LEFT/RIGHT determines to which side of the boundary the specified values will be located

19 Design Function Range Defintions CREATE PARTITION FUNCTION pf_baddate (date) AS RANGE LEFT FOR VALUES ( , , ) ; CREATE PARTITION FUNCTION pf_gooddate (date) AS RANGE RIGHT FOR VALUES ( , , ) ;

20 Design Scheme CREATE PARTITION SCHEME mypf_scheme AS PARTITION mypf_function [ ALL ] TO ( FG1, FG2, FG3,...,FGx ) ; ALL keyword : if used, only one FileGroup name is specified and all partitions are created on that FileGroup FileGroup list: must have at least as many entries as boundaries in the Function FileGroup list: the same FileGroup can be specified more than once FileGroup list: partitions from the Function are created on FileGroups in the order specified

21 Design How to a Table Create a table on a Scheme and load your data Use the Create Wizard in SSMS. s an existing table by stepping you through your choices. Can create the Function & Scheme as it goes Rebuild the Clustered Index on the Scheme rather than the FileGroup

22 Design Splitting, Merging and Switching SPLIT where we add an additional boundary value in the Function. The where the boundary value would have been placed is then Split into 2 based on the Left/Right definition MERGE where we remove a boundary value from the Function and 2 partitions become one SWITCH we change the definition of the Allocation Unit to belong to a different HOBT, effectively moving the to a different table

23 Design Sliding Window Loads ed Table Switching s (1) Split New, empty NOTE: Empty s at left & right to ensure no need for data movement

24 Design Sliding Window Loads Switching s ed Table (1) Split (2) Switch Load Load Table Load Data to Staging Table

25 Design Sliding Window Loads Switching s ed Table (1) Split (2) Switch Archive Archive Table

26 Design Sliding Window Loads Switching s ed Table (1) Split (2) Switch (3) Merge Archive Table Archive

27 Configure & Manage ing Demo

28 Performance & Pitfalls

29 Design Performance Choice of Key is Critical What filters/joins are most commonly used? Align non-clustered indexes with the Key What other tables are involved should they also be partitioned

30 Design Performance Achieving Elimination Have to include the column in the predicates

31 Performance & Pitfalls Re-use of the Scheme / Function Wrong choice of RANGE LEFT or RANGE RIGHT may cause unexpected data movement Merging an empty with a non-empty partition may also cause significant data movement Elimination MAY result in more optimised plans but not always. it depends

32 Performance & Pitfalls TRUNCATE permitted at the level Switching is only meta-data, right? Maintenance at the Level. E.g. Index Rebuilds Watch out for aggregates!

33 Performance & Pitfalls Demo

34 Questions?

35

36 Session Evaluation