EMC CLARiiON Database Storage Solutions: Microsoft SQL Server 2000 and 2005

Transcription

1 EMC CLARiiON Database Storage Solutions: Microsoft SQL Server 2000 and 2005 Best Practices Planning Abstract This technical white paper explains best practices associated with Microsoft SQL Server 2000 and SQL Server 2005 when deployed on the EMC CLARiiON CX and CX3 UltraScale series storage platforms. Storage considerations for database deployments are discussed including configuration, replication, and performance monitoring and turning. August 2007

2 Copyright 2004, 2006, 2007 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com All other trademarks used herein are the property of their respective owners. Part Number H601.4 Best Practices Planning 2

3 Table of Contents Executive summary...5 Introduction...5 Audience... 5 Terminology... 5 CLARiiON architectural overview...6 Hardware architecture... 6 CLARiiON storage configuration...8 Storage-array configuration considerations...8 Best practices... 8 Storage-array caching considerations... 8 Best practices... 9 Disk type considerations... 9 Best practices... 9 RAID type considerations... 9 Best practices Database file considerations Best practices Windows NTFS allocation unit size considerations Best practices CLARiiON disk alignment value considerations Best practices MetaLUN technology...12 Best practices CLARiiON iscsi support...12 Database partitioning and Virtual LUN technology...13 Database partitioning Virtual LUN technology Best practices Database replication and disaster recovery...15 SnapView MirrorView/S SAN Copy RecoverPoint Clones of mirrors Best practices Storage performance...17 Best practices Navisphere Analyzer Best practices Navisphere Manager statistics...20 Navisphere Quality of Service Manager Best practices Windows performance monitoring Conclusion...23 Best Practices Planning 3

4 References...24 Related documents Best Practices Planning 4

5 Executive summary This white paper documents recommendations and guidelines associated with the operation of EMC CLARiiON storage arrays and Microsoft SQL Server 2000 and 2005, with additional consideration for distance replication and disaster recovery scenarios, database partitioning, and LUN migration. General configuration guidelines and operational issues are discussed, with special attention given to typical database workloads. Environmental considerations of availability, scalability, and performance are also covered. This paper specifically focuses on longer distance and/or lower-cost scenarios where direct fiber connections are impractical. Introduction Microsoft SQL Server underpins many of the latest financial, manufacturing, ecommerce, and collaborative applications from leading third-party vendors including Microsoft. Many businesses have also developed in-house applications using SQL Server databases. As companies adopt SQL Server-based solutions for increasingly important business applications, it is critical that these solutions protect the integrity of both the data as well as the operational environment. EMC CLARiiON storage arrays, with their advanced information availability and replication features, are ideally suited for implementing SQL Server solutions. CLARiiON supports five to 480 disk drives, supports Fibre Channel or iscsi host connections, and accommodates SAN, NAS, and DAS environments. CLARiiON storage arrays are ideally suited for SQL Server solutions because they deliver high availability through a no-single-points-of-failure design; advanced information replication with multiple choices local or remote; synchronous or asynchronous, powerful storage management with EMC Navisphere ; and exceptional affordability by mixing ATA and Fibre Channel drive technology within the same system. The true differentiation of CLARiiON, however, can be found in its bulletproof data integrity handling. Maximizing the effectiveness of CLARiiON storage arrays in a SQL Server environment takes careful planning and understanding of database storage considerations. Understanding the database environment and required service levels all factor into proper database storage-system design. Replication, especially over longer distance and higher latency connections, exacerbates these issues. This white paper offers storage-array configuration guidelines and considerations based on years of database expertise with both Microsoft SQL Server and CLARiiON products. This paper is divided into seven sections: CLARiiON architectural overview CLARiiON storage configuration MetaLUN technology CLARiiON iscsi support Database partitioning and Virtual LUN technology Database replication and disaster recovery Storage performance Audience This white paper is intended for customers, EMC partners, SQL Server database administrators, storage architects, and storage administrators who require information about how the different CLARiiON replication products work and how they can provide a replication solution that fits customer needs. A general understanding of SQL Server and CLARiiON hardware and software will aid the reader. Terminology Advanced Technology-Attached (ATA): A standard interface for connecting storage devices such as hard disks and CD-ROM drives. Best Practices Planning 5

6 Bandwidth: The amount of data a storage system can process over time, which is measured in megabytes per second. CLARiiON array replication software: Optional software set offered for CLARiiON systems including SnapView, MirrorView, and SAN Copy, supporting efficient replication of data stored in CLARiiON systems. CLARiiON consistency: The logical concept of designating (and enforcing) a group of CLARiiON storage LUNs to be replicated by the replication software as a coherent set. The replication action is performed automatically for all the members in the set. Either all members will be replicated, or none will be replicated. Clone: A full binary copy of a storage element. Fibre Channel (FC): A technology for transmitting data between storage devices and computer devices at data rates of up to 4 Gb/s (and 10 Gb/s in the near future) on a dedicated network (optical fiber, or for short runs, coaxial cable or twisted pair). Logical unit number (LUN): A unique identifier used to identify logical storage objects in a storage system. Mirroring: A technique that allows data to be written to two duplicate disks simultaneously. If one of the disk drives fails, the system can switch to the other disk without any loss of data or service. Redundant Array of Independent Disks (RAID): A way of storing the same data in several places on multiple hard disks. I/O operations can overlap in a balanced way, thereby improving performance. Since multiple disks increase the mean time between failures, storing data redundantly also increases fault tolerance. A RAID storage system appears to the operating system to be a single logical hard disk device. Response time: The interval of time between submitting an I/O request and receiving a response. SnapView snapshots: A snapshot is a virtual LUN that allows a secondary server to view a point-in-time copy of a source LUN. You determine the point in time when you start a SnapView session. The session keeps track of the source LUN s data at a particular point in time. Table partitioning: Breaking a single database table into sections stored in multiple files. Throughput: The number of individual I/Os the storage system can process over time, which is measured in I/Os per second. Virtual LUN technology: A technique whereby data stored on a LUN is bitwise migrated to another LUN within the same storage system, without restricting access to the data. CLARiiON architectural overview This section presents an overview of the CLARiiON CX3 series hardware architecture. Various elements of the storage-system design are discussed, particularly as they relate to the effective implementation of Microsoft SQL Server 2000 and 2005 databases. Hardware architecture CLARiiON storage systems have been engineered for full redundancy, eliminating single points of failure. The arrays consist of two storage processors redundantly connected to dual-ported back-end Fibre Channel disk drives and high-capacity ATA drives. The following sections provide details of CLARiiON CX3 hardware features. The CLARiiON CX3 UltraScale storage systems contain one storage processor enclosure (SPE) and up to 16 4 GB disk-array enclosures (DAE2s) supporting up to 480 drives. The SPE houses two storage processors (SPs), redundant power supplies, and redundant cooling modules. Each DAE2 contains up to 15 disk drives and redundant Fibre Channel link modules, power supplies, and cooling fans. The first DAE2 and the SPE Best Practices Planning 6

7 are battery-protected with two standby power supplies (SPSs). Figure 1 depicts the CLARiiON CX3-80 UltraScale architecture. DAE2 modules can be dynamically added to the storage system without impacting availability or performance, thanks to Fibre Channel loop technology. This feature allows storage administrators the option of adding database performance or capacity without compromising the integrity or availability of operational systems. Within the DAE2, a switched topology is used to attach the drives. The loop switch enables drive fault isolation and advanced diagnostics. Each CLARiiON storage processor (SP) contains front-end Fibre Channel ports for the host connection, Fibre Channel ports for back-end disk connections, mirrored battery backed cache, and high-speed interconnects to the adjacent SP. A total of eight back-end Fibre Channel loops service up to 480 drives, providing high-bandwidth redundant access to every drive from each SP, in turn providing capacity and performance flexibility to any SQL Server solution, no matter how large. The cache in each SP is user-configurable in terms of memory size for read cache, memory size for write cache, cache block size, prefetch mode, and cache management policy. These features can be important in fine-tuning Microsoft SQL Server 2000 or 2005 installations. CLARiiON mirrored write cache guarantees the persistence of write data, even in the event of total power failure or the loss of any storage-system component, including one of the SPs. During normal operation, write data is mirrored to both battery-protected SP cache memory before the host I/O is acknowledged. In the event of any storage-system component failure, the mirrored write cache content from the still functioning SP is immediately destaged to disk and write caching is disabled until the failed component is replaced. In the event of a total power outage, the battery backup ensures that all write cache content that still needs to be destaged is immediately written to an area on the disk known as vault before the system powers down completely. There is no long-term dependency on the power reserve in the battery, and committed write data will never be lost regardless of the duration of power outage. This design exceeds the SQL Server requirement for battery-backed write cache. Storage Processor x8 CMI Storage Processor Fibre Channel Mirrored cache CPU CPU FC FC FC FC Blower SPS Power Supply Blower Blower Power Supply Blower SPS Fibre Channel Mirrored cache CPU CPU FC FC FC FC LCC LCC LCC LCC LCC LCC Figure 1. CLARiiON CX3 Model 80 storage-system architecture Best Practices Planning 7

8 CLARiiON storage configuration The importance of understanding and planning the storage system cannot be underestimated since the storage system affects SQL Server performance more than any other factor. Performance improvement depends heavily on hardware characteristics such as disk layout, I/O characteristics, disk allocation units, and storage array caching practices. Following general guidelines should suffice for most midrange database designs. Storage-array configuration considerations Consider the following best practices when configuring a CLARiiON array for SQL Server database. Best practices Be aware of the capacity and performance limitations of vault drives. Vault and array firmware drives are the first five drives of the first disk tray. These drives have slightly reduced capacity, and slightly lower available IOPs while performing array management or configuration tasks. It is preferable to store less performance critical data in these drives. Bind one hot spare for every 30 drives. A fully configured rack should have two to three hot spares, depending on database criticality and break/fix response times. Connect sufficient Fibre Channel ports into the storage network. Usually, this is as many front-end ports as are available; however, all eight ports may not be required for a given workload, and the extra expense may not be justified. MirrorView, if configured, supports both switch connection and sharing of host and mirror traffic across the same port. Use multiple paths for high availability and improved performance. Configure multiple paths between host servers and storage arrays using multiple HBAs and switches. Use at least two NICs or HBAs in each host. Install PowerPath software on all hosts, which provides failover and load balancing across available active paths. Load balancing I/O loads across multiple paths to the storage system ports avoids storage bottlenecks. PowerPath monitors data flow through all active paths. In case of a path failure, it removes the failed path and reroutes all data access through alternate live paths until the failed path is repaired and restored. Be aware of the impact of reserved LUN pool on database performance. Allocation of the reserved LUN pool used for snapshot cache and asynchronous mirroring can have significant ramifications on overall array and database performance. Understand the anticipated I/O load required for these functions and locate and size the reserve LUN pool accordingly. For more information about reserved LUN pools allocation considerations for CLARiiON storage arrays, refer to EMC CLARiiON Reserved LUN Pool Configuration Considerations Best Practices Planning at EMC.com and Powerlink. Storage-array caching considerations Onboard cache on CLARiiON storage array can greatly enhance the performance of SQL Server I/O operation. The CLARiiON storage arrays offer a wealth of options for enabling and tuning the storage-array cache. Read and write caching are independently configurable on each LUN in the array. Default array behavior is to enable read and write caching for all volumes. There is no default for array cache allocation. The following cache configuration best practices may deliver more efficient database performance from CLARiiON arrays. Best Practices Planning 8

9 Best practices Set the storage array cache block size to the default 8 KB. This setting matches the array cache block size to the standard SQL Server database page size of 8 KB. Allocate sufficient read and write cache on storage array. For more information about cache allocation considerations for CLARiiON storage arrays, refer to The RAID engine cache section in the EMC CLARiiON Best Practices for Fibre Channel Storage: CLARiiON Release 24 Firmware Update white paper at EMC.com and Powerlink. Enable write caching for all volumes. Disable write caching for read-only or non-database LUNs only if the write cache becomes saturated. Navisphere Analyzer can be used to monitor cache utilization and forced flush rates. Enable read and write caching for SQL Server transaction log volumes. Minimizing response time for transaction log writes allows SQL Server more concurrency. Enable write caching for RAID 5 database volumes. Write caching helps alleviate the effects of the RAID 5 lower write performance. Disable read caching for volumes with mostly random reads. Consuming storage processor cache resources for these read operations will not significantly improve data response times, and will detract from the performance of other volumes that are less random in nature. The exception to this guideline applies to LUNs that will see sequential access for backup operations. In this case, the read cache should be enabled during the backup operation. Enable read caching for volumes with frequent sequential data reads. Database table scans are sequential in nature and will benefit from read prefetching. Enabling read caching for volumes with small to medium indexes that are frequently accessed will also benefit from read prefetching. Disk type considerations Select a disk type based on performance, cost, and reliability requirements of a SQL Server application environment. Best practices Use FC drives for high-performance SQL Server applications. FC drives provides high reliability and high random read/write performance. FC drives are available in either 10k or 15k rpm models. Use the faster 15k rpm FC drives for OLTP type workloads with a large number of small random I/O reads and writes. Use ATA drives for low cost and large capacity. ATA drives cost less and provide large capacity. But ATA drives have a lower duty cycle and deliver less performance than FC drives. Because of lower random I/O performance, ATA drives are not suitable for OLTP type applications. ATA is ideal for storing aged data, replication, or backup copies of database files that are read or written to disk in large sequential I/Os. RAID type considerations Consideration should be given for the RAID type used in SQL Server database, transaction log and backup and restore volumes, as this decision will affect database performance, fault tolerance, and cost. RAID consists of multiple disk drives used primarily for improved I/O performance and large storage size. In CLARiiON, such a group of disk drives is referred as a RAID group. You can achieve optimal response time from the array subsystem by configuring the RAID levels to match your application I/O workload patterns. CLARiiON storage arrays offer multiple RAID level choices of RAID 0 striping, RAID 1 mirroring, RAID 1/0 striped mirrors, RAID 3 and RAID 5 striping with parity and RAID 6 striping with dual parity. For Best Practices Planning 9

10 more information about RAID configuration considerations for CLARiiON storage arrays, refer to the EMC CLARiiON Best Practices for Fibre Channel Storage white paper at EMC.com. Best practices Use CLARiiON hardware RAID instead of Windows server software RAID. Software RAID uses host processor cycles that can impact the performance of SQL Server applications. Use more disk drives in a RAID group to improve performance. Build RAID groups consisting of smaller disk drives rather than a few large drives. This enables more disks to work in parallel to read data, thus improving performance. Be aware of using large stripes (large RAID groups). It takes longer to rebuild a large parity-based stripe such as RAID 5 and RAID 6 LUNs. Instead use a striped metalun consisting of multiple smaller LUNs, which provides improved performance and shorter rebuild time. For more information on metaluns, refer to section titled MetaLUN technology in this document. Select a RAID type based on its performance, fault tolerance, cost and application work load. RAID 1/0 gives excellent random read/write performance. RAID 1/0 is ideal for OLTP environments with lots of small random read/writes. This RAID type also provides good fault tolerance since it can survive the failure of up to half of the disks, provided one disk in each mirror image pair survives. Use RAID 1/0 if more than 30 percent of the I/O is small random writes and if budget permits. RAID 5 gives excellent read performance, especially large sequential I/O. But it provides lower random write performance. It also delivers lower fault tolerance than RAID 1/0 since it can tolerate only one drive failure per RAID 5 LUN. Also, in the event of a drive failure, the time for the storage system to rebuild the content of the failed drive is longer than RAID 1/0. During the content rebuild of the failed drive, overall performance to the data will be impacted, though there will not be a loss of data access. RAID 5 costs less for the same storage capacity compared to RAID 1/0. RAID 6 provides higher level of data protection at the cost of lower performance than other RAID levels. Its storage capacity is N-2 due to additional parity data, whereas RAID 5 provides storage capacity of N-1. RAID 6 can survive two drive failures without losing data, providing an added level of data protection over RAID 5 which can survive only a single disk failure. However, RAID 6 requires additional parity calculation which can reduce overall write performance. RAID 6 also incurs more rebuild time than RAID 5. Choose RAID 6 if high availability is important and performance can take a modest hit. RAID 6 is ideal for Serial ATA (SATA) based drives to provide insurance against multiple drive failures that have a higher chance of occurring due to lower duty cycle and longer rebuild times of the dense SATA drives. For more information about RAID 6, refer to the white paper EMC CLARiiON RAID 6 Technology. RAID 6 requires FLARE release 26. Database file considerations Like any other database, SQL Server manages data using database files and transaction log files. The database files are frequently accessed with random reads and writes. Transaction log files, on the other hand, consist mostly of sequential write operations, with occasional reads of recently written transaction records. SQL Server tempdb database is read/write intensive, and is used for storing temporary tables, temporary stored procedures, and sub-queries, and for sorting aggregate operations. The following are best practice guidelines for choosing RAID types for SQL Server database, transaction log, and tempdb files. Best practices Spread database files on RAID group consisting of more disk drives. It is more I/O efficient to use a RAID group containing several small disks than a RAID group with few large disks. Best Practices Planning 10

11 Place SQL Server transaction log and database files on physically separate RAID groups. Placing SQL Server database and transaction log files on physically separate RAID groups avoids disk contention. Place transaction log files on RAID 1/0 volumes. RAID 1/0 volume provides excellent write performance as well as fault tolerance and is an ideal choice for the placement of sequentially write intensive SQL Server transaction log files. RAID 1/0 also rebuilds faster than RAID 5 in case of a disk failure. Place database files on RAID 1/0, RAID 5 or RAID 6 volumes. Choose RAID 1/0 volumes for OLTP type applications with highly active database files. Choose RAID 5 for random I/O only if writes are less than 30 percent of the workload. RAID 5 can be used for data warehouse type applications with mostly large data reads. Choose RAID 6 if high availability is important and the application is willing to tolerate reduced disk performance. Consider placing database files on metaluns for improved performance, ease of expandability, and fast rebuild. Place tempdb database on RAID 1/0 volume. RAID 1/0 provides excellent read/write performance and fault tolerance; place read/write intensive SQL Server tempdb database files on a physically separate RAID 1/0 volume. Preallocate tempdb volume to the expected maximum file size because extending tempdb while writing can degrade performance. Windows NTFS allocation unit size considerations Choosing the right NTFS allocation unit is as important as selecting right RAID type or adding more disks to your storage system to achieve the maximum SQL Server performance. What is a disk allocation unit? Windows Server organizes the hard disk based on the file system disk allocation unit, which consists of a group of contiguous sectors. The number of sectors in an allocation unit can be 4 KB to 256 KB. Choosing the right disk allocation unit is crucial because smaller read/writes on a large allocation unit can waste storage space while larger disk read/writes on a smaller allocation unit causes disk fragmentation resulting in slower file access on subsequent read/writes. Windows chooses the default allocation unit size depending on the volume size when you format the volume. However, you must choose the disk allocation unit size (8 KB, 16 KB, 64 KB, and 256 KB) based on your application s disk I/O workload characteristics for maximum performance. If the disk I/O characteristics of an application are not known, determine the size of the average disk I/O from the application by measuring the number of reads and writes per second from the storage subsystem using a performance monitor. The allocation size can be set when a partition is formatted using Microsoft Windows Disk Manager. Best practices Choose a 64 KB allocation unit size for SQL Server volumes. A 64 KB allocation size matches the CLARiiON internal storage structure size and alignment for optimal storage performance. For more information, refer to the File-system alignment section in EMC CLARiiON Best Practices for Fibre Channel Storage: CLARiiON Release 24 Firmware Update at EMC.com and Powerlink. CLARiiON disk alignment value considerations When a partition is created using Microsoft Disk Manager on CLARiiON disk, the partition is not normally aligned with the underlying physical disk. The misalignment causes more I/Os to the disk, resulting in reduced performance. The partition can be aligned properly using Microsoft Windows disk partition tool DISKPART.exe in Windows Server First create the partition using diskpart.exe with a specific alignment value set. The partition is then formatted using Microsoft Windows Disk Manager with a specific allocation size. For more information regarding partition alignment, refer to the Using diskpar and diskpart to Align Partitions on Windows Basic and Dynamic Disks white paper at EMC.com and Powerlink. Best Practices Planning 11

12 Best practices Create SQL Server disk partitions using the DISKPART.exe tool with the disk alignment value set to 64K for improved disk performance. MetaLUN technology CLARiiON metalun technology is designed primarily for the efficient use of storage capacity. It allows LUN concatenation for storage expansion and LUN striping for improved storage performance. MetaLUNs can consist of striped LUNs, concatenated LUNs, or a combination of both. LUNs in a metalun must be the same capacity but can be different RAID types, except in the case of RAID 6, where all RAID groups must be RAID 6. For additional information about the CLARiiON metalun capability, refer to EMC CLARiiON MetaLUNs: Concepts, Operations, and Management at EMC.com. MetaLUNs are managed through Navisphere Manager. MetaLUNs provide the following benefits for SQL Server storage: Performs striping or concatenation of database LUNs within the array instead through host-based software. Enables on-the-fly LUN expansion without any disruption to host applications. Allows the storage space to be expanded as needed instead of preallocating disk space for anticipated future growth. Allows quick expansion of an existing LUN by concatenating additional LUNs. The concatenated metalun provides instant capacity and allows mixed RAID types. Allows the creation of high-performance metaluns by striping a large number of disk spindles or concatenating another striped set to an existing stripe set. Allows multiple stripe elements. A metalun can be built of separate, striped components that are concatenated together. Adding a striped component to an existing metalun is as fast as concatenation, and offers the performance benefit of striping. For more information, refer to the MetaLUNs section in EMC CLARiiON Best Practices for Fibre Channel Storage: CLARiiON Release 24 Firmware Update. Provides higher availability. For example, an 8+1 RAID 5 group would lose all the stored data in any volume in the RAID group if two drives fail simultaneously. A metalun LUN compromised of two 4+1 RAID LUNs have logically the same capacity will not lose any data, or become inaccessible, if two separate drives, one in each sub-lun fail. Best practices Use a striped metalun for high performance. Striped metaluns provide excellent read/write performance, especially in a random I/O read and write environment. A striped metalun consists of multiple smaller LUNs and provides better performance and shorter rebuild time than a single LUN in a large RAID group with the same number of disks Use concatenated metaluns for quick expansion of existing storage space. Concatenating LUNs to form metaluns allows quick expansion without disruption to the application. But unlike striped metaluns, concatenating LUNs does not improve performance. CLARiiON iscsi support iscsi is a network protocol standard that allows the exchange of SCSI commands and data over IP networks. CLARiiON CX3 FC/iSCSI model 10, 20, and 40 array systems support Fibre Channel as well as iscsi storage, allowing the user to connect hosts to the FC SAN, IP network, or both. These FC/iSCSI combo storage systems allow customers to implement low-cost networked storage solutions for connecting hosts and EMC storage. iscsi can be used for consolidating remote hosts and storage systems to a SAN over IP networks. It also allows storage-based long distance replication and disaster recovery through the IP network using CLARiiON SAN Copy, SnapView, MirrorView/S or MirrorView/A. Best Practices Planning 12

13 Best practices Use 1 Gb/s iscsi HBAs capable of offloading TCP/IP processing on host servers. Offloading TCP/IP processing to HBAs improves the performance of hosts running CPU intensive SQL Server applications Use a 1 Gb/s dedicated private network between the host and storage system using 1 Gb/s capable iscsi HBAs and IP switches. For remote replication, use 1 Gb/s dedicated IP network between the storage systems. Use the diskpart.exe tool to align at 64 KB for improved performance when creating an iscsi LUN. iscsi is not recommended for large sequential I/O loads associated with streaming media and DSS workloads. For more information, refer to the Practice Considerations for CLARiiON CX3 FC/iSCSI Storage Systems at EMC.com and Powerlink. Database partitioning and Virtual LUN technology Database partitioning is a new feature in SQL Server 2005 that allows partitioning database tables across file groups to improve performance. Virtual LUN technology is a storage-based data migration feature provided by EMC that can be used for seamless movement of database tables or partitions within a CLARiiON storage system. Database partitioning SQL Server 2005 database partitioning is a technique used for splitting up very large database tables into smaller and more manageable individual tables. This allows managing a multi-terabyte database more efficiently at a more granular level on smaller tables. Further, these subtables can be distributed over several file groups across different RAID groups to improve performance. Partitioning can be done horizontally or vertically. In horizontal partitioning, groups of rows are distributed into different tables based on primary key value or date column. For example, using data and time the recent data can be stored in one set of subtables on a quarter-by-quarter basis while older data can be stored in different subtables on a year-by-year basis. This results in faster search and deletion of current or older data on smaller tables than on large tables. These tables can be load balanced by placing on separate disk volumes. Additionally, heavily used tables can be moved to faster storage and less active ones to slower storage through LUN migration (refer to Figure 2), allowing the database archiving requirements of Information Lifecycle Management (ILM) to improve performance and availability. Database partitioning is primarily intended for very large databases such as a data warehouse with hundreds of gigabytes of data. Partitioning provides the following advantages: Improves database query performance due to faster operations on smaller tables. Increases database availability. Backup and restore of individual tables can be done without interruption to the rest of the database. Simplifies database maintenance tasks, since the task needs to run only on a subset of the main table containing far fewer rows instead of the whole table. Allows load balancing to minimize I/O contention by distributing tables across several storage partitions. Provides the ability to move active data tables to faster storage partitions and less active aged data to slower and inexpensive storage partitions. Best Practices Planning 13

14 Figure 2. Table partitioning enabling online data management Virtual LUN technology EMC virtual LUN technology allows the seamless movement of LUNs within a CLARiiON storage system without disrupting the host applications. The movement of LUNs is achieved using CLARiiON LUN migration technology (refer to Figure 3). The entire process occurs with no disruption to host I/O requests to the source LUN. After LUN migration, the destination LUN will inherit the source LUN's identity (LUN ID, WWN, LUN name) and the original source LUN will be destroyed. All subsequent access will be directed to the new storage locations transparently. Migration can take place to a destination LUN of same or larger capacity, different RAID type, and drive types (FC vs. ATA). LUN migration allows data warehouse applications to improve performance and capacity utilization by moving the database table to inexpensive or underutilized disk drives in the array. For example, active database tables can be moved to faster Fibre Channel drives while older less active data can be moved to lower cost, higher density storage. Additionally, it allows selecting a different RAID type for the destination drive to improve performance as well as capacity utilization. LUN migration is managed from the Navisphere Management Suite. For additional information, consult the white paper EMC Virtual LUN Technology - A Detailed Review at EMC.com and Powerlink. Best practices Split large database tables into smaller tables and distribute them over different RAID groups to improve performance. Move heavily used database table partitions to faster Fibre Channel devices and less active table partitions to slower and less expensive ATA devices. Consider using metaluns for heavily used partition tables for improved performance. Best Practices Planning 14

15 Use virtual LUN technology instead of SQL Server 2005 migration methods to migrate database table partitions to tiered storage. SQL Server 2005 offers ways to migrate partitions, but it requires a substantial number of database table manipulations steps to migrate a partition. LUN Migration Older data is aged to highest density storage after one year Last quarter data is aged to higher density storage Figure 3. Depicting LUN migration between different tiers of storage within the same CLARiiON array Database replication and disaster recovery CLARiiON provides a full range of array-based as well as appliance-based software for replication and disaster recovery in a SQL Server environment that can operate without wasting the host processor cycles. The CLARiiON replication software, SnapView, allows the local replication while MirrorView/S, SAN Copy and RecoverPoint can be used for remote replication for load balancing, offline processing, or disaster recovery. SnapView SnapView is storage-based software that allows local replication by creating mountable point-in-time snapshots or full copy clones of the original production data LUNs. These snapshots and clones are available to backup and recovery solutions, and product testing and data migration solutions. Since SnapView snapshots incorporate unchanged data from production LUNs, backup of the snapshots can impact the performance of the production LUNs. SnapView also provides a rollback feature that allows restore of a point-in-time snapshot back to the original source LUN, for situations where the original LUN becomes corrupted or the original LUN needs to revert to an earlier point-in-time data point. SnapView clones are physically separate full volume copies of the original production LUN. Therefore, backup of the clones can be done without affecting the performance of the production LUN. SnapView can manage up to eight clones and eight snapshot copies of a LUN and also allows up to eight snapshots of each clone. For additional information, refer to the EMC SnapView for Navisphere Administrator s Guide on Powerlink. Best Practices Planning 15

16 MirrorView/S MirrorView/S is storage-based software that can copy data images of a production host LUN synchronously to secondary storage at a remote site for disaster recovery. Primarily targeted for short-distance replication for disaster recovery, MirrorView/S delivers continuous bidirectional information integrity and data availability across different sites. It can maintain a realtime mirror image of the production data at remote sites where these mirror images can be used to run production at those remote sites in case of a natural disaster to the production site. Bandwidth and latency of the MirrorView/S interconnect are critical to successful deployments with SQL Server. Sufficient bandwidth must exist to initialize the remote mirror and sustain active mirroring operation. Latency must remain low enough to prevent database and application bottlenecks that could ripple through to the end user. High bandwidth and low latency requirements restrict the use of MirrorView/S for long-distance replication. SAN Copy SAN Copy is storage-based software that can be used for copying data between CLARiiON and multivendor storage systems (including IBM, HP, Hitachi Data Systems, and Sun storage arrays) without host intervention. Once copied remotely, database backups are available for disaster recovery, restarting production on a remote site, or archiving. SAN Copy supports both full and incremental data copy. A full update copies the entire contents of the source LUN to the destination LUN. An incremental update copies only the changed data on the source LUN to the destination LUN. SAN Copy can be leveraged with SnapView and Replication Manager for creating disaster recovery solutions for applications. In this case SnapView clones provide local replication and recovery while SAN Copy is used for remote replication and recovery. Replication Manager coordinates with the host application to create the clone and manages the copy process to the remote system. For additional information, refer to the white paper EMC CLARiiON SAN Copy - A Detailed Review at EMC.com and Powerlink. RecoverPoint RecoverPoint is an out-of-band appliance-based replication solution, used for replication of data locally or remotely at another site thousands of miles away from the primary site. For local replication it uses Fibre Channel and for remote replication it uses the IP to send replicated data over a WAN to distant sites. RecoverPoint can significantly enhance the SQL Server disaster recovery process through local or asynchronous remote replication. Local recovery provides continuous data protection (CDP) by capturing every write transaction to a database and simultaneously transmitting data synchronously to a secondary storage device. CDP allows the recovery of the data to the exact instant corruption has occurred with full transactional consistency. Remote replication and recovery of data over any distance is through continuous remote replication (CRR). CRR is done asynchronously and it does conserve much bandwidth due to compressing the data before transmitting to the remote site. However, in the event of a production site failure due to a disaster, any data that is queued may be lost, resulting in backdated data at the remote disaster recovery site. RecoverPoint also offers a failover processing mode in which production can be failed over to the remote site. For additional information, refer to EMC RecoverPoint Family Overview - A Detailed Review and Improving Microsoft SQL Server Recovery with EMC RecoverPoint at EMC.com. Clones of mirrors CLARiiON clones of mirrors feature allows the creation of the clones of a LUN s mirror image on a primary or secondary site. It is beneficial to use the clones of mirrors instead of snapshots for backup copy or production run on the secondary site because using snapshots on secondary site normally puts load on the primary site. Also, failure of a source LUN will make its snapshots unusable. Because clones are fully populated binary copies of their source LUN on a secondary site, rather than the pointer-based and copy-onfirst-write model of snapshots, clones provide greater data protection and performance than snapshots. Clones of mirrors are created using SnapView software. Clones of mirrors require FLARE release 24 or later. Best Practices Planning 16

17 Best practices Use SnapView to replicate SQL Server database and transaction log volume snapshots on the local storage system. In case of database corruption, roll back SnapView snapshots to the production volumes. Note that the SnapView snapshot is not a separate copy of the primary LUN. Therefore a primary LUN failure can also fail the backup snapshots. Use MirrorView/S for short-distance (campus/metropolitan) replication. MirrorView/S in combination with SnapView and Replication manager can replicate database and transaction log volumes into remote site mirrors, which can then be used for restart database in a remote site in case of primary site failure. SQL Server disaster recovery configurations using MirrorView/S should take advantage of clones of mirrors for replication of mirror images. Because clones have much less of a performance impact than snapshots upon the remote copy and provide a higher level of protection against hardware failures. Back up the SQL Server transaction log at frequent intervals to a standby server using SAN Copy in combination with SnapView and Replication Manager. In case of a disaster, recover the database from transaction log. Use RecoverPoint Continuous Data Protection (CDP) for local replication. In case of database corruption, the entire database can be recovered to any given point in time from the secondary storage area. Use RecoverPoint Continuous Remote Replication (CRR) for long-distance replication where latency and low bandwidth of the link are a concern. During disaster recovery, the entire database and log volumes can be recovered to any given point in time from a remote site or use RecoverPoint failover mode to fail over the production database to the remote RecoverPoint site. It is also used for replicating data between heterogeneous storage environment such as CLARiiON to Symmetrix, or any other array vendor storage systems. Storage performance Good SQL Server database design practices include proper sizing of the I/O subsystem, primarily the diskdrive configuration comprising the data and log files. The I/O response time of database servers with directattached storage is almost completely determined by the disk-drive latencies. Bandwidth becomes a limiting factor in bulk load operations, backup and restore, and large sequential table scan operations. Choosing proper RAID type and enabling storage-array read/write caching can significantly improve response times. RAID type considerations and CLARiiON cache configuration guidelines are presented in the CLARiiON storage configuration section. A disk-bound I/O database subsystem can improve performance by simply adding drives to the storage array(s). Connection-bound (bandwidth limited) database servers must add additional storage arrays or storage connection paths to improve performance. Three principle items can have tremendous impact on a SQL Server storage subsystem: server memory, number of disks, and number of paths to the storage. Monitoring and tuning SQL Server database installations should be standard practice in test and production deployments. Performance can be measured using the Windows performance monitoring tool and CLARiiON Navisphere Analyzer. Best practices Make sure to have sufficient memory in the host server. System memory is very important to SQL Server and insufficient memory can strain the disk drives because server memory serves as the database s primary cache. The more memory available for database caching, the fewer I/O operations the storage subsystem will need to service. Insufficient memory also causes excess page swapping, putting additional load on the host processor. Best Practices Planning 17

18 Align partitions using the Microsoft Windows Diskpart utility. Create database partitions with alignment value set to 64K for improved disk performance. For more information, refer to the File-system Alignment section in the EMC CLARiiON Fibre Channel Storage Fundamentals white paper. Ensure that there is sufficient number of disk drives in the storage system. The number of disks that make up the database store determines the sustainable I/O rate the database can use without exceeding required latencies. Adding more drives and/or replacing current disks with faster drive in a RAID 1/0 or RAID 5 array will increase the I/O rate. Migrate data to high performance RAID types. Leverage CLARiiON LUN migration to migrate existing data on one RAID type to one with higher performance (for example, from RAID 5 to RAID 1/0). Ensure multiple paths of storage connections to the storage system. Additional storage connection path through SAN fabrics and/or additional storage arrays can be added for high availability and also to lighten the load of saturated storage components. Use well-designed database tables and well-tuned queries. Poorly designed database tables and queries can generate excess disk I/O requests that put load on the storage system. The SQL Profiler tool can be used to identify long running and frequently running queries. Tuning queries to minimize the amount of data returned can greatly improve SQL Server disk I/O performance. Adding the index to database tables can also dramatically improve query performance and thus reduce load on the storage. Ensure that the database volumes are not fragmented. Excessive disk I/O can be a symptom of a fragmented SQL Server database volume. If the disk volumes are fragmented, defragmenting database volumes will improve disk I/O performance, resulting in improved query performance. The best practice is to create files on contiguous space that is already defragmented. Improve storage performance using CLARiiON and Windows performance tools. Analyze and identify CLARiiON storage performance bottlenecks using Navisphere Analyzer and Windows performance monitor. Leverage Navisphere Quality of Service Manager (NQM) facility to establish workload performance prioritization policies to create the most optimal balance of system throughput for meeting challenging business requirements. Navisphere Analyzer Navisphere Analyzer is a browser-based performance analysis tool that is used for analyzing CLARiiON storage performance patterns. It shows charts and graphs of detailed realtime and historical information about the array, using the data collected from disks, LUNs, storage cache, storage processors, and SnapView sessions as shown in Figure 4 and Figure 5. The information from charts and graphs can be used for performance analysis and tuning. The following are best practice guidelines for storage performance tuning using Navisphere Analyzer and Navisphere Manager. Best practices Look into storage processor (SP) utilization and disk utilization. If SP utilization is higher than disk utilization, adding additional disk to RAID group won t improve performance. If SP % Utilization is higher than 70% consider balancing the storage processor workload. Move LUNs from the overutilized SP to the underutilized SP using Navisphere Manager If performance utilization is still high, consider reducing I/O demand on the storage processor. This can be done by reducing the number of application threads or number of users. If SP is not the performance bottleneck then consider tuning the disk. Look for LUNs with the highest % Utilization in the RAID group. If their Average Queue Length is 50% more than their peers or if their Response Time is consistently higher than other LUNs in Best Practices Planning 18

19 the group then check their disk % Utilization. If the disk utilzation is higher than 70%, then adding more disk drives to the RAID group will improve performance. Consider adjusting the storage cache to reduce the disk workload. If the % Used prefetches are less than 70%, then try reducing the prefetch multiplier If Read Cache Hit Ratio is below 50%, turn off read cache altogether. If there is excessive forced flushes, which indicates that the write cache is not large enough to accommodate the modified data, check Average Busy Queue Length and Queue Depth of LUNs and disks. An Average Busy Queue Length higher than Queue depth indicates bursty I/O. Adjust cache water marks in order to free more reserved cache to absorb write bursts. If a heavy work load is causing the performance degradation then adding more disks to the RAID group will improve performance. Figure 4. Navisphere Analyzer LUN statistics monitoring Best Practices Planning 19

20 Figure 5. Navisphere Analyzer MirrorView/A monitoring Navisphere Manager statistics Navisphere Manager provides the ability to monitor realtime statistics of many CLARiiON storage system components. The array statistics listed in Table 1 are frequently helpful in performance bottleneck identification and isolation. Table 1. Navisphere Manager statistics Array statistics Storage System Properties (Cache) Storage Processor Properties (Statistics) LUN Properties (Statistics) Disk Drive Properties (Statistics) Description Read and write cache hit ratios for each storage processor, and an indication of how much of the write cache is actively in use. Additional values for SP utilization and read/write throughput and bandwidth. Values for LUN utilization, read/write throughput and bandwidth, and stripe crossings. High values for stripe crossings generally indicate improper partition or file-system formatting. Disk-drive-specific values for utilization and read/write throughput and bandwidth. Navisphere Quality of Service Manager A critical SQL Server 2005 application in a multiple application environment sharing a common CLARiiON array may experience decreased performance due to storage resource contention. EMC Navisphere Quality of Service Manager (NQM) remedies this problem by allowing users to control storage resource for noncritical applications in order to improve performance of critical applications. This resource balancing is done by controlling the bandwidth, throughput, and response time thresholds of individual applications resource using customizable policies or according to time of day. Applications can be set to run at various Best Practices Planning 20

21 performance levels based on priorities at different times of the day. For example, a mission-critical SQL Server application can gain performance by limiting performance of other applications running on the same server. A backup application can be scheduled to run at night time with a specific performance level. Note that NQM does not improve overall storage performance; it only prioritizes performance among applications by giving more resources to a critical application than others. Also, NQM supports user application LUNs only; it may not provide resource control on private LUNs used by CLARiiON add-on applications. NQM provides following types of resource control modes: Limit mode - Allows the user to set not-to-exceed performance level (throughput, bandwidth or response time) for a group of LUNs used by a secondary application, thus freeing up resource for a high priority application. This will be useful in limiting a secondary application like a backup program from loading up an array while critical application like SQL Server is accessing the same array resources. Cruise Control mode Allows the user to set a specified performance level for a group of LUNs. This mode is useful in maintaining SQL Server application at a specified performance level while running with other applications. Fixed Queue Depth mode Allows the user to set the queue depth for a group of LUNs used by a specific application, there by controlling the amount of I/O used by the application. This is an advanced feature and should be used only if the user has in-depth knowledge of the storage performance. Controlling an application resource using NQM involves creating an I/O class and applies appropriate resource management policies to this class. Use the following steps to manage an application resource: 1. Create a resource I/O class for the application (refer to Figure 6 below). 2. Add application LUNs to the I/O class. 3. Set a performance limit level using Limit Control mode. 4. Set a performance target level using Limit Control mode or Cruise Control mode. 5. Schedule the I/O class to run at a specific time of the day. 6. Start the NQM for the specific resource I/O class and monitor the performance NQM requires FLARE release 24 or later. For additional information, refer to the Navisphere Quality of Service Manager (NQM) - Applied Technology at EMC.com. Best practices Use NQM to determine if the storage array is causing the performance bottleneck. This is done by monitoring the application performance at the array level using NQM Control Panel. Improve performance of SQL Server mission-critical applications by limiting bandwidth of other low priority applications using NQM Limit mode. For example, you can increase the bandwidth of a SQL Server application to 50 MB/s by limiting the bandwidth to other applications to a lower level. Improve performance of a mission-critical application by setting a specific bandwidth using NQM Cruise Control mode. For example, bandwidth of a SQL Server application can be maintained at a level of 50 MB/s using Cruise Control mode. This will result in lesser bandwidth available to other applications. Use Navisphere Quality of Service Manager integrated scheduler to apply different service levels at different times of the day for different applications. Best Practices Planning 21

22 For example, run SQL applications with high priority during day time and a backup application with high priority at night time. Figure 6. Navisphere Quality of Service Manager Control Panel Windows performance monitoring The Windows Performance Monitor allows administrators to view or collect realtime performance counter information on a wide variety of components, including SQL Server components, processor, memory, network, and disk subsystem. Each of these categories contains a set of counters that are used to measure specific items related to the category. For example, processor, disk, and memory counters can be used for analyzing disk bottlenecks. Interesting processor, storage, and disk counters for analyzing storage performance bottlenecks are provided in Table 2. Additionally, SQL Server provides a set of storage specific counters that can be used for analyzing SQL storage performance problems. Note that physical disk and logical disk performance counters must be enabled with the diskperf command prior to being collected. Table 2. Terminology for Windows performance monitoring % Processor Time Indicates what percentage of time the processor is busy. This counter can occasionally spike 100%; however the average value should not exceed 80%. A constant high value reading may be an indication of excessive page swapping due to insufficient system memory. % Disk Time Shows the percentage of time that the selected disk drive is busy servicing read or write requests. An indication of how often a given disk/volume is busy. If this value is greater than 70%, it shows a disk I/O bottleneck. Best Practices Planning 22