T-SQL Performance Issues What you should know to avoid, find and resolve them Claire Mora
T-SQL Performance Issues What you should know to avoid, find and resolve them Preparing your environment Assessing the situation Resolving the issues Testing your solution
Preparing your environment Reduce distractions, take convenient regular breaks Try to match the production system Gathering metrics Get a live database backup (if possible) Use controlled data with sensible scaling Ensure you have the right tools
1 2 3 4 Server: "DEV05", Standard Edition (64-bit) SP1 v10.50.2550.0, Collation=Latin1_General_CI_AS Server Memory Used = 1541.77mb out of 8192mb Row Sample = 1,000,000 CPU speed test...29.0s TempDB [SIMPLE Recovery] ROWS = 0.49gb "D:\SQLServerData\tempdb.mdf" LOG = 0.68gb "D:\SQLServerData\templog.ldf" Server Insert...1.53s Add primary key...0.68s Server Read (single, uncached)...0.38s Server Read (single, cached)...0.37s Server Read (parallel, cached)...0.41s Client Read One...0.08s Client Read All...4.72s Server Delete rows...1.94s Server Drop Table...0.01s LocalTest [SIMPLE Recovery] ROWS = 0.39gb "D:\SQLServerData\LocalTest.mdf" LOG = 4.10gb "D:\SQLServerData\LocalTest_log.ldf" Server Insert...1.95s Add primary key...0.92s Server Read (single, uncached)...0.70s Server Read (single, cached)...0.74s Server Read (parallel, cached)...0.69s Client Read One...0.08s Client Read All...4.66s Server Delete rows...1.62s Server Drop Table...0.00s
if (object_id('dbo.test') is not null) drop table Test create table Test ( gid bigint not null,rowdate date not null,rowvalue float not null,rowname nvarchar(max) not null ) ;with z(v) as (select 0 from (values (0),(0),(0),(0),(0),(0),(0),(0),(0),(0)) t(v)), RowGenerator (Offset) as (select row_number() over (order by (select 0)) from z z1,z z2,z z3,z z4,z z5,z z6) insert into Test (gid,rowdate,rowvalue,rowname) select top(500000) gid=offset,rowdate=dateadd(hour,offset,'20000101'),rowvalue=rand(hashbytes('sha1',convert(varchar,offset)))*10000,rowname=replicate('x',rand(hashbytes('sha1',convert(varchar,offset)))*100) from RowGenerator select top(500000) * from Test if (object_id('dbo.test') is not null) drop table Test
Assessing the situation Identifying the issues DMV stats, statistics, profiles, schema Execution plans, I/O, messages USE of resources (CMIN) Common issues
Graphical Execution Plans
Text Execution Plans
A tiny part of XML Execution Plans
I/O and Timing messages
Performance Report
Halloween Hell Using Table [#Test] logical reads = 1,904 Using Table [Worktable] Scan count = 1 logical reads = 908 450 rows 100% WITH z 1x450r 35% Clustered Index Insert : Insert OBJECT:(tempdb.#Test), SET:(tempdb.#Test.Offset = Expr1038) 1x450r 46% Table Spool : Eager Spool 1x450r Top TOP EXPRESSION:((0)) 1x450r Nested Loops : Left Anti Semi Join OUTER REFERENCES:(Expr1038) 1x500r Filter WHERE:(Expr1038>=(1) AND Expr1038<=(500)) 1x500r Top TOP EXPRESSION:(CASE WHEN (500) IS NULL OR (500)<(0) THEN (0) ELSE (500) END) 1x500r 1x500r Segment 0x0r Compute Scalar DEFINE:(Expr1037=(0)) 1x500r 1% Nested Loops : Inner Join NO JOIN PREDICATE 1x50r Nested Loops : Inner Join NO JOIN PREDICATE 1x5r Constant Scan 5x50r Constant Scan Sequence Project : Compute Scalar DEFINE:(Expr1038=row_number) 50x500r Constant Scan 500x50r 16% Cl u stered Index Seek OBJECT:(tempdb.#Test AS t), SEEK:(t.Offset=Expr1038), WHERE:(tempdb.#Test.Offset as Line 14 CPU time = 0 ms, elapsed time = 2 ms. Common Issues
Parallel Plans Using Table [TabTest] Scan count = 1 logical reads = 1,854,807 Using Table [Worktable] Scan count = 5 logical reads = 71,440 Using Table [Worktable] 500000 rows 100% INSERT INTO TabTest (TestId, Cat, Com) 1x500000r 44% Cl u stered Index Insert : Insert OBJECT:(TabTest.PK TabTest 8CC331607FB5F314), SET:(TabTest. 1x500000r 1% Parallelism : Gather Streams ORDER BY:(Expr1014 ASC) 6x500000r 49% Sort ORDER BY:(Expr1014 ASC) 0x0r Co mpute Scalar DEFINE:([Expr1014]=INSERTED.[TestId] as [ins].[testid]+convert_implicit(bigint,u 6x500000r 4% Parallelism : Distribute Streams 1x500000r Top TOP EXPRESSION:((0)) 1x500000r N e sted Loops : Left Anti Semi Join WHERE:([TabTest].[TestId] as [tt].[testid]=(inserted.[testi 1x500000r Nested Loops : Inner Join NO JOIN PREDICATE 1x5r Constant Scan VALUES:(((1)),((2)),((3)),((4)),((5))) 5x500000r Cl ustered Index Seek OBJECT:(INSERTED AS ins), SEEK:(ins.Act=(4)) ORDERED FORWAR 500000x0r 1% Clustered Index Scan OBJECT:(TabTest.PK TabTest 8CC331607FB5F314 AS tt) TrigTest Line 4 CPU time = 9111 ms, elapsed time = 11327 ms. Common Issues
Common Issues Halloween Hell Data bubbles Parallel plans RBAR Missing Indexes Parameter sniffing
Common Issues cont. Using the wrong types causing implicit conversions Foreign keys without the necessary indexes for deletes Recursive CTEs Over complex queries Functions on rows or without schema binding Triggers Bad choice of index keys Disabled foreign Keys or indexes Untrusted constraints Out of date statistics XML Queries on rows Badly designed schema
Resolving the issues Planning your approach be aware of your table row counts and structure Dividing the tasks Schema change solutions Library of techniques
Library of techniques Divide and Conquer
Query Hints MAXDOP 1 hint RECOMPILE hint OPTIMIZE FOR hint Library of techniques
Row generators select x from (values (0),(0),(0),(0),(0),(0),(0),(0),(0),(0)) t(x) ;with z(x) as (Select 0 from (values (0),(0),(0),(0),(0),(0),(0),(0),(0),(0)) t(x)),rg(x) as (select 0 from z z1,z z2,z z3,z z4) Select 0 from rg ;with z(x) as (Select 0 from (values (0),(0),(0),(0),(0),(0),(0),(0),(0),(0)) t(x)),rg(x) as (select 0 from z z1,z z2,z z3,z z4) Select row_number() over (order by (select 0)) from rg ;with z(x) as (Select 0 from (values (0),(0),(0),(0),(0),(0),(0),(0),(0),(0)) t(x)),rg(x) as (select 0 from z z1,z z2,z z3,z z4) Select top(500) row_number() over (order by (select 0)) from rg Library of techniques
Row Generators o These can be used to do the following Generating rows Duplicating rows Transposing Working with arrays Removing duplicate keys with record based binary fields Library of techniques
Library of techniques Divide and Conquer Query hints MAXDOP 1 RECOMPILE OPTIMIZE FOR Row Generators Hash Comparisons Library of techniques
Recommended reading 1 Temporary Tables with post indexes Creating the index after the table data is filled means less juggling around so is much quicker. Quirky Update Used when you have to reference prior rows in the same result set like with totaling. Apply join (cross, outer) Used with XML queries or when joining to the same table or filtering individually per row. Sargability Determines index usage based on operators used. Library of techniques Exists Detects the presence of a row so can work faster than the in operator. Conditional Filters or Ordering Short circuiting unnecessary filters or ordering using constant comparisons can speed things up a lot. Binary fields Used for complex data structures where wide keys over-inflate the database, many items using the same key.
Recommended reading 2 Sensible indexes Used to avoid the foreign keys slow deletion problem and to prevent slow sorts. Schema Binding For functions which have no table access this can dramatically speed up the execution. Using functions instead of views Allows filtering the results so can be far faster than views in complex queries. Date operations Always use date arithmetic instead of strings to build dates as it is much faster and is compatible with international dates. List processing Used for delimited strings when building or splitting to allow passing lists as parameters. XML is the fastest method and should be used where lists are required. Just beware implicit conversions and parameter sniffing. Library of techniques
Testing your solution Data validity and Consistency Resource usage / stress testing / Multi-user testing Version testing and production testing Peer reviews before + after comparisons
T-SQL Scripts
SQLPrep
Multi process testing
T-SQL Performance Issues What you should know to avoid, find and resolve them Be SMART use PART Preparing your environment Assessing the situation Resolving the issues Testing your solution Y use this method?
Want to know more? I do!... See me for a chat and to pick up a mini SQL cheat sheet Quick query?! Tweet me @csqls Loopy long questions? Email me claire.mora@csqls.com