Data Partitioning. For DB Architects and Mere Mortals. Dmitri Korotkevitch

Transcription

1 Data Partitioning For DB Architects and Mere Mortals Dmitri Korotkevitch

2 Please silence cell phones

3 Explore Everything PASS Has to Offer FREE ONLINE WEBINAR EVENTS FREE 1-DAY LOCAL TRAINING EVENTS VOLUNTEERING OPPORTUNITIES LOCAL USER GROUPS AROUND THE WORLD ONLINE SPECIAL INTEREST USER GROUPS PASS COMMUNITY NEWSLETTER FREE ONLINE RESOURCES BUSINESS ANALYTICS TRAINING BA INSIGHTS NEWSLETTER

4 Session Evaluations 3 Your feedback is important and valuable. ways to access Submit by 5pm Friday November 6 th to WIN prizes Go to passsummit.com Downloadthe GuideBook App and search: PASS Summit 2016 Follow the QR code link displayed on session signage throughout the conference venue and in the program guide

5 Dmitri Korotkevitch 20+ Years in IT 15+ Years working with SQL Server Microsoft Data Platform MVP Microsoft Certified Master Author: Pro SQL Server Internals Expert SQL Server In-Memory OLTP

6 Agenda Architecture-focused discussion about data partitioning Partitioning in the single database Distributed data partitioning Going non-relational

7 Challenge: Inaccurate Statistics

8 Challenge: Mixed Workload Support

9 Challenge: Disaster Recovery / RTO

10 Challenge: Index Maintenance

11 Challenge: Storage Cost

12 Partitioned Tables [Orders2016] [FGFastStorage] >= [Orders2015] [Orders2014] <

13 Partitioned Views [Orders2016] [FGFastStorage] [Orders2015] [Orders2014]

14 Comparing approaches Partitioned Tables Enterprise and Developer editions only 1,000 / 15,000 partitions max Moving data between filegroups keeping table online requires coding Statistics on the table level Same schema and indexes No partition-level onlineindex rebuild prior SQL Server 2014 Partitioned Views All editions of SQL Server 255 tables/partitions max Datacan be moved between filegroups online(enterprise Edition) (* it depends) Separate statistics per table Different schema and indexes per table. Can combine In-Memory OLTP, B-Tree and columnstore data Onlineindex rebuild within the table (Enterprise Edition) (* it depends) 14

15 Challenges PARTITIONED TABLES PARTITIONED VIEWS Inaccurate statistics Does not solve Solves Different schema and indexing Not supported Supported Different compression Supported Supported Piecemeal restore and partial backup and availability Supported Supported Per-partition online index rebuild SQL Server Supported Development overhead None Significant Tiered storage and online data movement Requires development Supported Sliding window and metadata-level data purge Supported Supported 15

16 Tiered Storage and Mixed Workload «Hot» Data

17 Moving Data Demo

18 Moving Data Online SQL Server Enterprise Edition SQL Server R2 Enterprise Edition MOVINGPARTITION TO DIFFERENT FILEGROUP Requires development ONLINE INDEX REBUILD WITH LOB COLUMNS IN THE INDEX Supported Not supported ONLINE INDEX REBUILD WITHOUTLOB COLUMNS INTHE INDEX Supported Supported Non-Enterprise Editions N/A Not supported Not supported MOVING DATA TO DIFFERENT DISK ARRAY Supported in every edition (adds fragmentation)

19 Mixed Approach [Orders2016] [FGFastStorage] [Orders2015] [Orders2014] < Regular Tables

20 Mixed Approach [Orders2016] [FGFastStorage] [Orders2015] < [Orders2014] Partitioned Tables

21 Mixed Approach [Orders2016] [FGFastStorage] [Orders2015] < [Orders2014] Partitioned View

22 Mixed Approach [Orders2016] [FGFastStorage] [Orders2015] < [Orders2014] SCH-M lock during ALTER TABLE ADD CHECK CONSTRAINT (INDEX SCAN)

23 What about In-Memory OLTP? [Orders2016] In-Memory OLTP FG Clients Articles [Orders2015] [Orders2014] < Native Compilation

24 What about In-Memory OLTP? [Orders2016] tmp09 In-Memory OLTP FG tblupdated tbldeleted Trigger [Orders2015] [Orders2014] <

25 What about In-Memory OLTP? [Orders2016] tmp09 In-Memory OLTP FG tblupdated tbldeleted Trigger [Orders2015] [Orders2014] <

26 Moving In-Memory OLTP Data Demo

27 Linked Servers

28 Linked Servers Demo

29 Ugly Workaround = count(*) where Archived = 1; > 0 begin = (select StoreId as [@Id] where Archived = 1 for xml path); insert into #RemoteOrders @StopDate; end select... from ( select.. from dbo.orders union all select.. from #RemoteOrders )

30 Stretch Database Pros: Transparency Transactional consistency Cons: Limited control price, Price, PRICE!!!!

31 Outside of Relational Model Raw Data: Sensor Readings Geo Locations Row-by-row processing => NoSQL Reportable Data: Events / Actions Trips Joins; aggregations; analysis; reporting => Relational DB

32 Sounds Too Good To Be True Partitioning affects execution plans More at: Pro SQL Server Internals No uniqueness across all the data Several other restrictions / incompatibilities

33 We Covered Partitioning in the single database Distributed data partitioning Going non-relational

34 Any Questions? Thank you very much! Blog:

35 Session Evaluations 3 Your feedback is important and valuable. ways to access Submit by 5pm Friday November 6 th to WIN prizes Go to passsummit.com Downloadthe GuideBook App and search: PASS Summit 2016 Follow the QR code link displayed on session signage throughout the conference venue and in the program guide

36 Thank You Learn more from Dmitri Korotkevitch