Advanced SQL Programming and Optimization Everything you wanted to know about Stored Procedures!
About Soaring Eagle Since 1997, Soaring Eagle Consulting has been helping enterprise clients improve their overall system performance at the database tier, arguably the most volatile and critical component of distributed application architecture. Our clients range in size from fledgling startups through Fortune 100 companies and leading financial institutions. Soaring Eagle has been a leader in managed services, database architecture, performance and tuning databases, while promoting mentoring and training all over the world for over a decade and a half. Many of our employees, and partners have written books, speak at seminars about leading edge technologies. We have expertise in all business tiers, financial; health, manufacturing, government agencies and many ecommerce businesses. Whatever your business needs are we can help improve performance! Consulting Performance & Tuning Data Performance Management Emergency Triage Performance & Security Audits Staff Augmentation Project management Database architecture Scalability assessment and planning Training Onsite/Web based Microsoft Sybase Oracle APM Six Sigma Managed Services Remote Database Management Performance management Emergency db Service Proactive mitigation Problem notification Problem resolution Software Application Performance Management Database performance management Database Security 2-43 Soaring Eagle Consulting 8/19/2013
Acknowledgements Microsoft SQL server, SSMS are trademarks of Microsoft Inc. This presentation is copyrighted by Soaring Eagle Consulting, August 7, 2013 This presentation is not for re-sale This presentation shall not be used, modified, or redistributed without express written consent of Soaring Eagle Consulting, Inc. Solarwinds Ignite is registered trademark of Solarwinds Inc. 3-43 Soaring Eagle Consulting 8/19/2013
Topics Review stored procedure coding recommendations Define appropriate error handling, return status and parameter techniques Understand compilation issues Examine performance factors 4-43 Soaring Eagle Consulting 8/19/2013
SP Coding Standards and Conventions Follow normal standard coding guidelines as you would for any programming language Use explanatory comments Use indentation to improve readability Develop structured methods of handling errors within stored procedures Develop a method for maintaining versions of stored procedure source code 5-43 Soaring Eagle Consulting 8/19/2013
Structured Error Handling in Stored Procedures There are two valid options: Try catch blocks Check for errors after every SQL statements take appropriate error handling steps Check @@error value Always use the return statement with status codes when procedure encounters an error Always check the return status of called stored procedures Set up defaults for all parameters and perform parameter checks at the beginning of procedure 6-43 Soaring Eagle Consulting 8/19/2013
Returning Procedure Status Every stored procedure automatically returns an integer status value Zero is returned on successful completion -1 through -99 are returned for SQL Server detected errors Use a return statement to specify a return value greater than or equal to 0 or less than -99 The calling program can set up a local variable to receive and check the return status 7-43 Soaring Eagle Consulting 8/19/2013
Returning Procedure Status (Cont d) create proc procedure_name [ (parm_name datatype = default_value [output] [,... ] ) ] as SQL Statements return [integer_status_value] [execute] [@status_var = ] procedure_name [[parm_name = ] expression [output] [, ] 8-43 Soaring Eagle Consulting 8/19/2013
Procedure Creation /* procedure sets an error status on error */ create proc titles_for_a_pub (@pub_name varchar(40) = null ) as if @pub_name = null return 15 if not exists (select * from publishers where pub_name = @pub_name) return -101 select title from publishers p join titles t on p.pub_id = t.pub_id where pub_name = @pub_name return 0 9-43 Soaring Eagle Consulting 8/19/2013
Return Status Example Procedure Usage /* check for status and report errors */ declare @status int exec @status = titles_for_a_pub 'New Age Books' if @status = 15 print 'Invalid Syntax' else if @status = -101 print 'No publisher by that name found' 10-43 Soaring Eagle Consulting 8/19/2013
SQL Server Status Codes Following is a list of return status codes currently in use by SQL Server Status Code Meaning 0 Successful return -1 Missing object referenced -2 Datatype mismatch error -3 Process chosen as deadlock victim -4 Permission error -5 Syntax error -6 Miscellaneous user error -7 Resource error, such as out of space -8 Non-fatal internal problem (bug) -9 System limit reached -10 Fatal internal inconsistency (bug) -11 Fatal internal inconsistency (bug) -12 Table or index corrupted -13 Database corrupt -14 Hardware error 11-43 Soaring Eagle Consulting 8/19/2013
Default Parameter Values Stored procedure parameters can be assigned default values if no value is supplied during execution You can improve your stored procedure code by defining defaults for all parameters create proc procedure_name (parameter_name datatype = default_value [,...]) as SQL Statements [return [status_value]] 12-43 Soaring Eagle Consulting 8/19/2013
Default Parameter Values (Cont d) Procedure Creation Example /* check for a pub_name before executing query */ create proc titles_for_a_pub (@pub_name varchar(40) = null) as if @pub_name = null begin print Pass in the pub_name as a parameter return end select t.title from publishers p join titles t on p.pub_id = t.pub_id where pub_name like @pub_name + % return 13-43 Soaring Eagle Consulting 8/19/2013
Executing with Parameters At execution time, parameters may be specified by position or by name If passed by name, parameters can be passed in any order [exec[ute]] procedure_name [[@parm_name = ]expression] [,... ] 14-43 Soaring Eagle Consulting 8/19/2013
Executing with Parameters (Cont d) Procedure Creation Example create proc myproc (@val1 int, @val2 int, @val3 int) as... go /* parameters passed by position here */ exec myproc 10,20,15 /* parameter passed by name here */ exec myproc @val2 = 20, @val1 = 10, @val3 = 15 15-43 Soaring Eagle Consulting 8/19/2013
Notes on Parameters Once you have started passing parameters by name, all subsequent parameters must be passed by name If you want to skip any parameters and have them take default values, you will need to pass parameters by name unless they are the last parameter(s) in the procedure In programming environments, passing parameters by name is more flexible and self-documenting than passing parameters by position 16-43 Soaring Eagle Consulting 8/19/2013
Stored Procedure Recompilation The SQL Server optimizer caches a query plan based on parameters passed at the time the query plan was generated. This procedure could conceivably generate two distinctly different query plans based upon the values supplied for @low and @high Consider the following procedure create proc range_value (@low int, @high int) as select sum (total_sales) from titles where price between @low and @high return 17-43 Soaring Eagle Consulting 8/19/2013
Stored Procedure Recompilation (Cont d) Subsequent executions of the procedure will use any available procedure query plan To guarantee the proper query plan for each execution, create the stored procedure with the with recompile option create proc range_value (@low int, @high int) with recompile as select sum (price) from titles where price between @low and @high return 18-43 Soaring Eagle Consulting 8/19/2013
Stored Procedure Recompilation (Cont d) To generate a new query plan for a specific execution use the with recompile option when executing the procedure execute range_value 10, 200 with recompile To generate new query plans for all procedures dependent on a specific table, execute sp_recompile table_name 19-43 Soaring Eagle Consulting 8/19/2013
Recompiling Stored Procedures Creating the proc with the with recompile option causes a new query plan to be generated for each execution. This creates a lot of overhead with the optimizer The effect of using with recompile on stored procedure execution will generate a new query plan for the current execution, but does not affect any existing query plans in cache The query plan used for subsequent executions is indeterminate 20-43 Soaring Eagle Consulting 8/19/2013
Recompiling Stored Procedures Procedure query plans are not automatically recompiled if indexes are added or statistics are updated Run sp_recompile on the affected table If indexes associated with the query plan are dropped, all procedure query plans will be automatically recompiled The only ways to flush all existing query plans from cache are to: Drop and recreate the stored procedure Run dbcc freeproccache (clears ALL procedure cache) Cycle the SQL Server 21-43 Soaring Eagle Consulting 8/19/2013
Other Compilation Issues create proc get_data (@flag tinyint, @value int) as if @flag = 1 begin select * from titles where price > @value exec get_titles @value end else begin select * from salesdetail where qty < @value exec get_sales_detail @value end 22-43 Soaring Eagle Consulting 8/19/2013
Alternatives to Recompile (Cont d) On first execution, the optimizer will generate a query plan for ALL select statements based upon the passed in parameters, regardless of the conditional branching Note: This could result in the wrong query plan being generated for one of the select statements Recommendation: If this type of stored procedure must be written, have it call multiple stored procedures (as opposed to creating it with recompile) 23-43 Soaring Eagle Consulting 8/19/2013
Calling Stored Procedures from Transactions A rollback inside a stored procedure can lead to unexpected results because it does not abort the batch /* transaction in the batch */ begin tran insert tally values (1) if @@error!= 0 rollback tran else execute the_proc insert tally values (3) if @@error!= 0 rollback tran else commit tran Continued Next Page 24-43 Soaring Eagle Consulting 8/19/2013
Calling Stored Procedures from Transactions (Cont d) Rolls back to outermost tran in calling batch Subsequent statements in batch are processed What values are inserted in the tally table? How many transactions are active during the procedure? After the procedure returns? /* proc has a tran also */ create procedure the_proc as begin tran proc_tran insert tally values (2) if @@error!= 0 rollback tran else commit tran return 25-43 Soaring Eagle Consulting 8/19/2013
Stored Procedures and Transactions: Notes SQL Server notes the transaction nesting level before calling a stored procedure. If the transaction nesting level when the procedure returns is different from the level when executed, SQL Server will display the following message: Transaction count after EXECUTE indicates that a COMMIT or ROLLBACK TRANSACTION statement is missing 26-43 Soaring Eagle Consulting 8/19/2013
Stored Procedures and Transactions: Notes (Cont d) This message indicates that transaction nesting is not synchronized Because a stored procedure does not abort the batch on a rollback tran, a rollback tran inside the proc could result in a loss of data integrity if subsequent statements are executed and committed A rollback tran rolls back all statements to the outermost transaction, including any work performed inside nested stored procedures that have not been fully committed (i.e., @@trancount > 0) A commit tran within the stored procedure only decrements @@trancount by one 27-43 Soaring Eagle Consulting 8/19/2013
Stored Procedures and Transactions: Guidelines Develop a consistent error handling strategy for failed transactions or other errors that occur within transactions Implement this strategy consistently across procedures/applications Implement transaction control in nested stored procedures Check whether the procedure is being called from within a transaction before issuing a begin tran Because rollback tran from a procedure does not abort the batch calling the procedure, follow these guidelines: Procedures should make no net change to @@trancount Issue a rollback tran only if the stored procedure issues the begin tran statement 28-43 Soaring Eagle Consulting 8/19/2013
How To: Called or Stand-Alone Procedure Template /* proc to demonstrate no net change to @@trancount ** but rolls back changes within the proc ** VERY IMPORTANT: return an error code ** to tell the calling procedure rollback occurred */ create proc p1 as declare @trncnt int select @trncnt = @@trancount -- save @@trancount value if @trncnt = 0 -- transaction has not begun begin tran p1 -- begin tran increments nest level to 1 else -- already in a transaction save tran p1 -- save tran doesn t increment nest level Continued Next Page 29-43 Soaring Eagle Consulting 8/19/2013
How To: Called or Stand-Alone Procedure Template /* do some processing */ if (@@error <> 0) -- or other error condition begin rollback tran p1 -- rollback to savepoint, -- or begin tran return 25 -- return error code -- indicating rollback end /* more processing if required */ if @trncnt = 0 -- this proc issued begin tran commit tran p1 -- commit tran, -- decrement @@trancount to 0 -- commit not required with -- save tran return 0 /* successful return */ Continued Next Page 30-43 Soaring Eagle Consulting 8/19/2013
Using Temporary Tables in Stored Procedures Temporary tables must be created before they can be referenced within a stored procedure A single stored procedure can create and reference a temp table as long as the creation statement is executed before any SQL statements which reference the table If a called-procedure references a temporary table or a regular table created externally, a temporary table with the same name and structure no longer has to exist at the time the stored procedure is created 31-43 Soaring Eagle Consulting 8/19/2013
Temporary Table Performance Tips Keep temporary tables as small as possible, vertically and horizontally Select only required columns, rather than select * Consider creating indexes on temp tables within your stored procedure If the temporary table is of sufficient size and is going to be accessed multiple times, it may be cost effective to create an index on it Always drop temporary tables as soon as possible Watch out for mysterious periodic slowdowns if you are creating very large temporary tables: the server may be checkpointing tempdb 32-43 Soaring Eagle Consulting 8/19/2013
Indexes on Temporary Tables create proc p1 as select title_id, type, pub_id, ytd_sales into #temp_titles from titles where price between $8 and $10 create clustered index tmp on #temp_titles(pub_id) select sum(ytd_sales) from #temp_titles where pub_id ='1324' Continued Next Page 33-43 Soaring Eagle Consulting 8/19/2013
Indexes on Temporary Tables (Cont d) select min(ytd_sales) from #temp_titles where pub_id = '4324' return Question Will the stored procedure use the index? -YES! 34-43 Soaring Eagle Consulting 8/19/2013
Stored Procedure Debugging Techniques Write it as a batch first To get the syntax right on a stored procedure, write small parts of it as a batch first, then store the procedure once the whole operation starts working Get showplan output with recompile showplan output are generated by the optimizer, which is only operating when a procedure is recompiled. To see the effect of different parameters on optimization, create the procedure with recompile, then drop and recreate the procedure without it when you go into production 35-43 Soaring Eagle Consulting 8/19/2013
Temp Procedures SQL Server provides the capability of creating temporary stored procedures A stored procedure with a # in front of its name is a temporary procedure accessed only by the current connection A stored procedure with a ## in front of its name is a temporary procedure accessible by other connections They both go away when the connection is lost With SQL Server, you should not create a temporary stored procedure. Instead use the sp_executesql stored procedure or exec (@string) 36-43 Soaring Eagle Consulting 8/19/2013
Procedure Optimization Object references are resolved at execution time The query plan is generated at execution time Long procedures will still run faster than in-line SQL, because there will be fewer network packets sent to the server 37-43 Soaring Eagle Consulting 8/19/2013
Summary Use proper error handling and return status methods Watch out for rollback in a transaction 38-43 Soaring Eagle Consulting 8/19/2013
Lab: Stored Procedures Part 1 Set statistics io and display actual plan, then run the following queries and compare the query plans and I/O select count(id) from pt_sample_ciidnck where key2 between 0 and 500 select count(id) from pt_sample_ciidnck where key2 between 35000 and 50000 Write a stored procedure called procn where N is your user number as follows create proc procn (@lo int = null, @hi int = null) as if @lo = null or @hi = null begin 39-60 Soaring Eagle Consulting 8/19/2013
Lab: Stored Procedures print Syntax: procn 1,500 return end select * from pt_sample_ciidnck where key2 between @lo and @hi return Make sure showplan and statistics io are on and execute your stored procedure as follows, comparing query plans and total I/O for each execution exec procn 0, 500 exec procn 35000, 50000 exec procn 35000, 50000 with recompile exec procn 0, 500 40-43 Soaring Eagle Consulting 8/19/2013
Lab: Stored Procedures Create a second version of procn as procn_2 with the with recompile option Set statistics io, statistics time on, look at these and the graphical plan. Execute procn and procn_2 as follows exec procn 35000, 50000 exec procn_2 35000, 50000 exec procn_2 0, 500 exec procn 0,500 exec procn 0,500 with recompile 41-43 Soaring Eagle Consulting 8/19/2013
Lab: Stored Procedures Compare query plans, I/O, and compile, cpu, and elapsed times Part 2 Create a proc that creates a temporary table with an index inside the procedure, then execute a select statement to make use of the index (don't force an index). Execute and look at query plan and statistics. Now rewrite the proc to force the index and review the statistics 42-43 Soaring Eagle Consulting 8/19/2013
Thank you! Questions? Jeff Garbus jeff@soaringeagle.guru 813-641-3434 43-43 Soaring Eagle Consulting 8/19/2013