Designing and Developing your Database for Application Availability

Transcription

1 Designing and Developing your Database for Application Availability Presented to the Melbourne CBD SQL PASS Chapter. Saxons - Level 8, 500 Collins Street. Melbourne, Australia 12:30 14:00, July 26 th 2010 Charley Hanania, QS2 AG B.Sc (Computing), MCP, MCDBA, MCITP, MCTS, MCT, Microsoft MVP: SQL Server Senior Database Specialist 1

2 My Background Now: Microsoft MVP: SQL Server Database Consultant (again, and very happy) at QS2 AG Formerly: Production Product Owner of MS SQL Server Platform at UBS Investment Bank Technical Team Lead ITIL v3 Certified SQL Server Certified since 1998 On SQL Server since 1995 Version 4 on OS/2 IT Professional since 1992 PASS Chapter Leader Switzerland Regional Mentor Europe European PASS Conference Lead Event Speaker 2

3 Contact Info Website: <- Slides here Twitter: Blog: Linked-in: 3

4 Agenda General Barriers to Application Availability Standard High Availability Methodology General Standard - Clustering One up on the Standard Database Mirroring Decreasing component start-up time Instant File Initialisation Partitioning your Database Infrastructure Eureka! Partitioning for Availability (PfA) Basic PfA Methodology Code Samples Questions 4

5 General Barriers to Availability Environmental Complexity Simplistic Network Topology View: Slightly more realistic view: And that s just scratching the surface of the Peripherals! Think about the layered complexity of the supporting software and drivers! 6

6 Standard HA Methodology Keep it up for as long as possible. Increase redundancy Decrease Single Points of Failure Bring it back as quickly as possible Decrease the dependency sequence Decrease component start-up time 7

7 General Standard - Clustering Keep it up for as long as possible. Increase redundancy Additional Servers waiting in readiness Increased redundancy = increased cost Increased redundancy = decreased utilisation Decrease Single Points of Failure Virtual IP Addresses Virtual Network Names Additional Network Cards etc Storage Bring it back as quickly as possible Decrease the dependency sequence Server is Started SQL Instance is an exclusive Cluster Resource Decrease component start-up time Instant File Initialisation App Database isn t useable until Primary Data File is opened and recovered Standby Primary 8

8 Database Mirroring We could discuss the architecture, benefits and uses of DB Mirroring, but I m told you re mainly architects and developers, and the doors here aren t locked If you d like to discuss DB Mirroring, we can chat after the session 9

9 Decreasing Component Startup Time INSTANT FILE INITIALISATION (Another DBA point, but one that is Golden and not to be missed!) Data and log files are first initialized by being filled and with zeros when created or restored. When SQL Server Starts, the tempdb database is recreated. Application databases do not come online until tempdb has. If large, then startup times can be quite long! Action: Ensure that instant file initialisation is set (Windows: SE_MANAGE_VOLUME_NAME right) The group that the SQL Server Service Account is in needs to be added to the Perform Volume Maintenance Tasks security policy, in the Local Security Policy Tool. I ve seen this reduce SQL Server Startup from 8 minutes to 1 second (max) 10

10 Decreasing Component Startup Time PARTITIONING DATABASE COMPONENTS Msdn lists the following (summarised) Hardware Partitioning Multiprocessors that enable multiple threads of operations. RAID (redundant array of independent disks) devices A table striped across multiple drives can typically be scanned faster than the same table stored on a single drive. Horizontal Partitioning Divide tables into multiple tables containing the same number of columns, but fewer rows. Partitioning data horizontally based on age and use is common. Vertical Partitioning Divide tables into multiple tables containing fewer columns through normalization and row splitting. 11

11 Additional Partitioning Options Logical Schema Partitioning Used mainly for securing and grouping objects by department, role, application etc Physical Partitioning Of Files and Filegroups Used mainly for performance Files placed on multiple different data volumes Multiple files created for round-robin writes and extent allocations 12

12 Eureka! Partitioning for Availability Each database has an area similar to a disk s boot record For a database this resides in the Primary data file If this file is corrupt or unavailable, the database will not be loaded and opened regardless of other data files A good practice is to not put any objects in this data file, and to put all your tables and objects into secondary data files. Within the DB you can distribute objects into files to achieve one or more of the following Enable an application to continue working when other data files are missing or corrupt by grouping needed objects together Restore the most critical files for your application system to start up first, allowing partial functionality of your application to continue without issue. 13

13 Open Discussion Which ways could we split the adventureworks db, focussed on client facing / downstream application functionality, criticality and startup sequence for Availability? 14

14 Adventureworks Schema

15 PfA Methodology Conceptual extension from Logical Schema Partitioning and Physical File Partitioning: 1. Don t put user defined objects in the Primary FileGroup i. Allows the Primary Filegroup to load faster 2. Schema Separate user defined objects i. For clarity of purpose and operational use ii. iii. Try not to use dbo schema (good design/operational practice) DBA s will schematize objects in a similar fashion to what was in the Adventureworks sample. For PfA you should look to map your schema s around your application s core functionality and use cases and criticality 3. Create Filegroups for each Schema i. Multiple if you want to partition historical data horizontally etc ii. Name them in a fashion that allows easy understanding of what is in them, or how they should be used. 16

16 PfA Methodology(2) Conceptual extension from Logical Schema Partitioning and Physical File Partitioning: 4. Design your application to modularise its data access needs i. This is critical to segmenting your application to specific data ii. files/filegroups. This includes stored procedure and function internals, use try/catch within sp s for example to decide whether rollback or other functionality is needed when a filegroup is offline. iii. The key flows could be to either contain data access within the necessary filegroups, or determine whether a function can be used to catch that an object is in an offline filegroup and skip it / do something else / roll back, or Catch Msg 8653, Level 16, State 1, Line 3 * and do as per point above. 5. Document application s modules, with emphasis on criticality and data dependencies. i. Allows stepped sequencing of apps by criticality during start up *[Tambs] While the database is in a state of partial availability, the filegroups that remain online can support queries. However, queries that depend on data that resides in filegroups that are offline return error 8653: Msg 8653, Level 16, State 1, Line 3 The query processor is unable to produce a plan for the table or view 'X' because the table resides in a filegroup which is not online. 17

17 USE [master] CREATE DATABASE [PfA] ON PRIMARY ( NAME = N'PfA_Primary',FILENAME = N'E:\MSSQL10.SQL2008INST01\MSSQL\Data\PfA_Primary.mdf',SIZE = 400MB ), FILEGROUP [PfA_FG1] ( NAME = N'PfA_F1_FG1',FILENAME = N'E:\MSSQL10.SQL2008INST01\MSSQL\Data\PfA_F1_FG1.ndf',SIZE = 300MB ), ( NAME = N'PfA_F2_FG1',FILENAME = N'E:\MSSQL10.SQL2008INST01\MSSQL\Data\PfA_F2_FG1.ndf',SIZE = 300MB ), FILEGROUP [PfA_FG2] ( NAME = N'PfA_F1_FG2',FILENAME = N'E:\MSSQL10.SQL2008INST01\MSSQL\Data\PfA_F1_FG2.ndf',SIZE = 300MB ), FILEGROUP [PfA_FG3] ( NAME = N'PfA_F1_FG3',FILENAME = N'E:\MSSQL10.SQL2008INST01\MSSQL\Data\PfA_F1_FG3.ndf',SIZE = 300MB ) LOG ON ( NAME = N'PfA_log',FILENAME = N'F:\MSSQL10.SQL2008INST01\MSSQL\Logs\PfA_PfA_log.LDF',SIZE = 100MB ) Adding Files and File Groups to a Database USE [PfA]; SELECT * FROM sys.database_files; SELECT * FROM sys.filegroups; 18

18 USE [PfA]; CREATE SCHEMA [Sales]; CREATE SCHEMA [Production]; CREATE SCHEMA [Person]; -- moving tables from dbo schema to -- [Production] and [Sales] schemas ALTER SCHEMA [Sales] TRANSFER [dbo].[customer]; -- Note that triggers move as well... ALTER SCHEMA [Production] TRANSFER [dbo].[billofmaterials]; ALTER SCHEMA [Person] TRANSFER [dbo].[contactinsert]; ALTER SCHEMA [Person] TRANSFER [dbo].[contact]; * Use Synonyms to help the transition from dbo native schema to proper schema usage -- change stored proc definitions to focus on right schemas ALTER PROC [dbo].[contactinsert] BIGINT,@VvcSalutation VARCHAR(4) = NULL,@VnvcForename NVARCHAR(100),@VnvcMiddlename NVARCHAR(100) = NULL,@VnvcSurname NVARCHAR(100),@VvcJobTitle NVARCHAR(100) = NULL,@VvcCompany NVARCHAR(100) = NULL,@VnvcAddressLine1 NVARCHAR(100),@VnvcAddressLine2 NVARCHAR(100) = NULL,@VnvcAddressLine3 NVARCHAR(100) = NULL,@VvcPostCode VARCHAR(12),@VnvcCity NVARCHAR(50),@VnvcState NVARCHAR(50) = NULL,@VinCountryID INT,@Vvc Address VARCHAR(100),@VvcBusinessPhone VARCHAR(25),@VvcBusExt VARCHAR(10) = NULL,@VvcFacsimile VARCHAR(25) = NULL,@VvcMobilePhone VARCHAR(25) = NULL,@VvcPassword VARCHAR(20) ) AS INSERT INTO [Person].[Contact] ([ContactID],[Salutation],[Forename],[Middlename],[Surname],[JobTitle],[Company],[AddressLine1],[AddressLine2],[AddressLine3],[PostCode],[City],[State],[CountryID],[ Address],[BusinessPhone],[BusExt],[Facsimile],[MobilePhone],[Password]) VALUES (@VbiContactID,@VvcSalutation,@VnvcForename,@VnvcMiddlename,@VnvcSurname,@VvcJobTitle,@VvcCompany,@VnvcAddressLine1,@VnvcAddressLine2,@VnvcAddressLine3,@VvcPostCode,@VnvcCity,@VnvcState,@VinCountryID,@Vvc Address,@VvcBusinessPhone,@VvcBusExt,@VvcFacsimile,@VvcMobilePhone,@VvcPassword); 19 Create and Allocate Objects to Schemas

19 CREATE TABLE [Sales].[SalesInvoice] ( [SalesInvoiceID] INT IDENTITY(1,1) PRIMARY KEY CLUSTERED,[Amount] MONEY,[CustomerID] INT,[DateInvoiced] DATETIME,[DatePaid] DATETIME ) ON [PfA_FG1]; Creating tables on a filegroup is easy as per the code sample. Moving existing tables can be a little more complicated: Use Create index (clustered) with Drop Existing Rebuild indexes etc to move them It may take many iterations to get the sequencing and code here right, but its worth it in the long run. 20 Allocate an Object to a Filegroup

20 ADO.NET Code Example ( This code example uses ADO.NET 2.0 (SqlClient). The following parameters are used in the example. strconn is the database connection string that includes the failover partner. strcmd is the command, such as Transact-SQL or a stored procedure, to execute. iretryinterval is the number of seconds to wait (sleep) between retries. imaxretries is the number of times to retry the command before failing. Example connection string ( strconn ) Data Source=SQLA\INST1;Failover Partner=SQLB\INST1; Initial Catalog=DBMTest;Integrated Security=True using System.Data; using System.Data.SqlClient; using System.Threading; I expect that the code here will only need slight Adjustments for use with Clustering Connection string would use virtual name Timeout retry delay would cater for longer startup/failover bool ExecuteSQLWithRetry_NoResults(string strconn, string strcmd, int iretryinterval, int imaxretries) { // Step 1: Enter the retry loop in case an error occurs for (int iretrycount = 0; iretrycount < imaxretries; iretrycount++) { try { // Step 2: Open the database connection conn = new SqlConnection(strConn); conn.open(); // Step 3: Execute the command if (null!= conn && ConnectionState.Open == conn.state) { cmd = new SqlCommand(strCmd, conn); cmd.executenonquery(); // Step 4: If there were no errors, clean-up & return success return true; } } catch (Exception ex) { // Step 5: Catch Error on Open Connection or Execute Command // Error Handling to be added here: ex } 21 Retry Logic (1)

21 Continued finally { // Step 6: Clean up the Command and database Connection objects try { // Clean Command object if (null!= cmd) cmd.dispose(); cmd = null; // Clean up Connection object if (null!= conn && ConnectionState.Closed!= conn.state) conn.close(); conn = null; } catch (Exception ex) { // Error handling to be added here } } // Step 7: If the maximum number of retries not exceeded if (iretrycount < imaxretries) { // Step 7a: Sleep and continue (convert to milliseconds) Thread.Sleep(iRetryInterval * 1000); } } // end for loop: iretrycount < imaxretries // Step 7b: If the max number of retries exceeded, return failure return false; } 22 Retry Logic (2)

22 SELECT s.data_space_id as Id,g.name as Filegroup_name,s.name as Logical_name,physical_name,g.is_default AS [IsDefault],g.is_read_only AS [ReadOnly],CAST( ISNULL ( ( SELECT SUM(gs.size)*CONVERT(float,8) FROM sys.database_files gs WHERE gs.data_space_id = g.data_space_id ), 0 ) AS float ) AS [Size] FROM sys.filegroups AS g INNER JOIN sys.master_files AS s ON (s.data_space_id = g.data_space_id) WHERE s.type = 0 AND s.database_id = db_id() AND s.drop_lsn IS NULL; Query Returning Files, Filegroups and Sizes From Partial Database Availability (Danny Tambs, May 2007) 23

23 Links and Resources AdventureWorks Database Diagram Whitepaper: Partial Database Availability (Danny Tambs, May 2007) Whitepaper: Partitioned Table and Index Strategies Using SQL Server 2008 (Ron Talmage, March 2009) 24

24 Questions? 25