Indexing survival guide for SQL 2016 In-Memory OLTP. Ned Otter SQL Strategist

Transcription

1 Indexing survival guide for SQL 2016 In-Memory OLTP Ned Otter SQL Strategist

2 About me SQL Server DBA since 1995 MCSE Data Platform Passionate about SQL Server Obsessed with In-Memory

3 Agenda Editions Indexes Recovery Temporal Maintenance Troubleshooting

4 Editions, as of SQL 2016/SP1 Max In-Mem/ Edition Enterprise Standard Web Express Columnstore OS limit 32 GB/32 GB 16 GB/16 GB 352 MB/352 MB

5 Standard Edition (32 GB) Feature Per db Per instance In-Memory Yes No

6 Standard Edition (32 GB) Feature Per db Per instance In-Memory Yes No Columnstore Depends Depends

7 Standard Edition (32 GB) Feature Per db Per instance Columnstore/In-Mem tbl Yes No

8 Standard Edition (32 GB) Feature Per db Per instance Columnstore/In-Mem tbl Yes No Columnstore/HD tbl No Yes

9 ROW STRUCTURE

10 Row structure HEADER Begin TS End TS Idx Ptr Data

11 Row structure HEADER 8060 Begin TS End TS Idx Ptr Data

12 Row structure in row/off row 8060??? in-row off-row

13 Row structure in row/off row 8060??? in-row off-row DISK TABLE: DYNAMIC

14 Row structure in row/off row 8060??? in-row off-row DISK TABLE: DYNAMIC MEM TABLE: STATIC*

15 SYNTAX

16 DURABILITY = SCHEMA_AND_DATA = PK SCHEMA_ONLY = ONE INDEX

17 Syntax CREATE TABLE [dbo].[inmem] ( [col1] [int] IDENTITY(1, 1) NOT NULL,[col2] [char](100) NOT NULL,INDEX IX_InMem_col2(col2) ) WITH (MEMORY_OPTIMIZED = ON,DURABILITY = SCHEMA_AND_DATA)

18 Syntax ALTER TABLE/ADD INDEX ALTER TABLE/ALTER INDEX ALTER TABLE/DROP INDEX

19 Consistent terms avoid confusion ALTER TABLE <tblname> ADD INDEX <ixname> [NONCLUSTERED] (col1); ALTER TABLE <tblname> ADD INDEX <ixname> [NONCLUSTERED] HASH (col1) WITH (BUCKET_COUNT = 64);

20 Consistent terms avoid confusion ALTER TABLE <tblname> ADD INDEX <ixname> [NONCLUSTERED] (col1); ALTER TABLE <tblname> ADD INDEX <ixname> [NONCLUSTERED] HASH (col1) WITH (BUCKET_COUNT = 64); RANGE HASH

21 Syntax ALTER TABLE dbo.mytable ADD INDEX IX_mytable_CustomerID(CustomerID), IX_mytable_OrderID(OrderID)

22 DEMO

23 Indexes Range Hash Columnstore

24 Characteristics of RANGE indexes INEQUALITY DYNAMIC PAGES!= FIXED LOCK/LATCH FREE

25 Characteristics of HASH indexes EQUALITY STATIC PAGES = FIXED BUCKETS/ ROUNDING

26 Indexes Attribute HASH RANGE Optimized for Equality seeks and inserts Inequality seeks and ordered scans

27 Indexes Attribute HASH RANGE Optimized for Equality seeks and inserts Inequality seeks and ordered scans Fixed size? can be rebuilt No

28 Indexes Attribute HASH RANGE Rows returned in ORDER Not possible Possible (key order)

29 Indexes Attribute HASH RANGE Rows returned in ORDER Single index for ASC/DESC Not possible (table scan) n/a Possible (key order) No

30 Indexes Attribute HASH RANGE Query must use ALL key columns Yes No

31 Indexes Attribute HASH RANGE Query must use ALL key columns Yes No Can be modified after table creation

32 Indexes Attribute HASH RANGE Query must use ALL key columns Yes No Can be modified after table creation FILLFACTOR No No

33 Indexes Attribute HASH RANGE Query must use ALL key columns Yes No Can be modified after table creation FILLFACTOR No No FRAGMENTATION No No

34 FK/UNIQUE/PRIMARY Condition Disk In-Mem FK references UNIQUE idx YES NO

35 Max key length Index type Max key length HASH RANGE None* 2500 bytes

36 Indexes COVERING LOGGING* MIN INDEXES MAX INDEXES*

37 ALTER TABLE

38 ALTER TABLE Can be parallel/logs metadata only Executes OFFLINE SCHEMABINDING (column changes) Memory requirements

39 Indexes: In row/off row CREATE TABLE dbo.inmem_very_long_row (pkcol INT IDENTITY PRIMARY KEY,col2 VARCHAR(8000) NOT NULL,col3 NVARCHAR(4000) NOT NULL,col4 NVARCHAR(2000) NOT NULL WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA)

40 ALTER TABLE ALTER TABLE dbo.inmem_very_long_row ADD INDEX IX1 (pkcol, col4) (4) (2000)

41 ALTER TABLE ALTER TABLE dbo.inmem_very_long_row ADD INDEX IX1 (pkcol, col4) (4) (2000) off-row

42 ALTER TABLE ALTER TABLE dbo.inmem_very_long_row ADD INDEX IX1 (pkcol, col4) (4) (2000) in-row

43 ALTER TABLE Serially executed and fully logged Key column movement Add/drop COLUMNSTORE index Add/alter variable length LOB Create new off-row column

44 COLUMNSTORE (CCI)

45 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes

46 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes No

47 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes No ALTER TABLE, columns Yes

48 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes No ALTER TABLE, columns Yes No

49 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes No ALTER TABLE, columns Yes No ALTER TABLE, indexes Yes

50 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes No ALTER TABLE, columns Yes No ALTER TABLE, indexes Yes Yes (inline)

51 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes No ALTER TABLE, columns Yes No ALTER TABLE, indexes Yes Yes (inline) Native compilation n/a

52 In-Memory Clustered Columnstore Item On-disk In-mem Primary copy of data Yes No ALTER TABLE, columns Yes No ALTER TABLE, indexes Yes Yes (inline) Native compilation n/a No

53 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes

54 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No

55 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes

56 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes No*

57 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes No* REBUILD Yes

58 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes No* REBUILD Yes Bucket only

59 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes No* REBUILD Yes Bucket only Filtered NC only

60 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes No* REBUILD Yes Bucket only Filtered NC only No

61 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes No* REBUILD Yes Bucket only Filtered NC only No Vertipaq optimization Yes

62 In-Memory Clustered Columnstore Item On-disk In-mem Archival compression Yes No REORGANIZE Yes No* REBUILD Yes Bucket only Filtered NC only No Vertipaq optimization Yes No

63 In-Memory Clustered Columnstore Item On-disk In-mem Parallelism Yes

64 In-Memory Clustered Columnstore Item On-disk In-mem Parallelism Yes Interop

65 Why is the number and placement of containers absolutely critical? CONTAINER (FOLDER) DATA/DELTA FILES

66 Recovery process MEMORY-OPTIMIZED TABLES CONTAINER CREATE files COPY data STREAM data

67 Recovery process MEMORY-OPTIMIZED TABLES CONTAINER CREATE files COPY data STREAM data CREATE indexes

68 Recovery process MEMORY-OPTIMIZED TABLES CONTAINER 1 CONTAINER 2 CONTAINER 3 CONTAINER 4

69 Recovery events OFFLINE/ONLINE SQL restart RESTORE Server boot READ_ONLY READ_WRITE READ_COMMITTED_SNAPSHOT ON/OFF

70 Potential recovery issues INDEXES* LOB CONTAINERS FCI/AG (durable)

71 TEMPORAL

72 DEMO

73 Temporal Tables Temporal In-Mem

74 Temporal Tables Temporal In-Mem History on-disk

75 Temporal Tables Temporal In-Mem Staging In-Mem

76 Temporal Tables Temporal In-Mem Staging In-Mem old versions

77 Temporal Tables Temporal In-Mem Staging In-Mem old versions 8% of temporal

78 Temporal Tables Temporal In-Mem Staging In-Mem old versions async 8% of temporal History on-disk

79 Temporal Tables Temporal In-Mem old versions Staging In-Mem Default IDX ValidFrom/ValidTo async History on-disk

80 Temporal Tables Temporal In-Mem old versions Staging In-Mem Default IDX ValidFrom/ValidTo SELECT * FROM table WHERE ValidFrom = async History on-disk

81 Temporal Tables Temporal In-Mem old versions Staging In-Mem Default IDX ValidFrom/ValidTo SELECT * FROM table WHERE ValidFrom = async History on-disk

82 Temporal Tables Temporal In-Mem old versions Staging In-Mem Default IDX ValidFrom/ValidTo SELECT * FROM table WHERE col1 = 5 async History on-disk

83 Temporal Tables Temporal In-Mem old versions Staging In-Mem Default IDX ValidFrom/ValidTo SELECT * FROM table WHERE col1 = 5 async History on-disk

84 Temporal Tables SQL 2016/SP1 Temporal In-Mem old versions Staging In-Mem Custom IDX col1 Default IDX ValidFrom/ValidTo SELECT * FROM table WHERE col1 = 5 async History on-disk

85 Temporal Tables SQL 2016/SP1 Temporal In-Mem old versions Staging In-Mem Custom IDX col1 Default IDX ValidFrom/ValidTo SELECT * FROM table WHERE col1 = 5 async History on-disk

86 Temporal custom indexing SQL 2016/SP1 TF Multiple indexes Add/drop indexes Dynamic table name Scripting/failover sys.sp_xtp_flush_temporal_history

87 INDEX MAINTENANCE

88 Index maintenance FRAGMENTATION FILLFACTOR REORGANIZE

89 Index maintenance HASH: REBUILD BUCKETS

90 Index maintenance, Columnstore REORGANIZE

91 TAIL/vDELTA RG ROW GROUPS

92 Index Maintenance, Columnstore REORGANIZE ** TAIL ** sys.sp_memory_optimized_cs_migration

93 Index maintenance, Columnstore COMPRESSION DELAY

94 STATISTICS Manually created Auto created Auto updated (compatibility level) Manually updated (UPDATE STATISTICS) Native modules

95 Troubleshooting

96 Troubleshooting HASH indexes sys.dm_db_xtp_hash_index_stats Avg. chain length/bucket count Dupes/collisions Searching on subset of key columns Inequality predicates

97 Native compilation issues EXEC

98 Native compilation issues natively_compiled_proc_slow_parameter_passing Issue Named parameters Mismatched datatypes xevents reason named_parameters parameter_conversion

99 Best practices Foreign keys Bucket count Chain length Update stats/recompile Native compilation (parameters)

100 unsupported features FULLTEXT FILTERED ASCENDING/DESCENDING COMPUTED COLUMNS INDEXED VIEWS INCLUDEd columns

101 SQL vnext > 8 indexes per table NC rebuild enhancement Indexes on computed columns

102 Parallelism INDEX SCANS HEAP SCANS INTEROP SAMPLED STATS ALTER TABLE*

103 Agenda Editions Indexes Recovery Temporal Maintenance Troubleshooting

104 e: Indexing survival guide for SQL 2016 In-Memory OLTP Ned Otter SQL Strategist