EMC Solutions for Microsoft Exchange 2003 CX Series iscsi

Transcription

1 EMC Solutions for Microsoft Exchange 2003 CX Series iscsi Best Practices Planning Guide Abstract This white paper presents the best practices for optimizing the performance for Exchange 2003 storage configuration on EMC CLARiiON via iscsi. February 2007

2 Copyright 2006 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com All other trademarks used herein are the property of their respective owners. H2651 EMC Solutions for Exchange 2003 CX Series iscsi 3

3 Table of Contents Introduction...5 Audience... 5 Terminology... 5 General environment recommendations...6 Exchange Best Practices... 6 Networking Best Practices... 7 Operating System Best Practices... 7 Backup and Restore Best Practices CLARiiON Best Practices Replication Manager Best Practices Appendix: A User Mid-Sized Enterprise Building Block Design Storage Layout Appendix: B...17 Connecting iscsi Initiators to CX Targets EMC Solutions for Exchange 2003 CX Series iscsi 4

4 Introduction It is important to plan an Exchange solution that can grow while maintaining optimum performance, high availability, and disaster recovery. This document is meant to be a resource guide for optimizing the performance for Exchange 2003 storage configuration on EMC CLARiiON via iscsi. It also provides a step-by-step process of planning the storage configuration for Microsoft Exchange 2003 for a single server and multiple servers. Audience The intended audience for this paper is IT administrators and system engineers who have an interest in implementing Microsoft Exchange 2003 using EMC CLARiiON systems. It is assumed that the reader has a general knowledge of Microsoft Exchange, Active Directory, and EMC CLARiiON features and terminology. Terminology Term ACTIVE DIRECTORY DISK VOLUME iscsi (INTERNET SCSI) iscsi INITIATOR iscsi TARGET RAID RAID 1 RAID 5 RAID 1_0 RAID GROUP SP Description An advanced directory service introduced with Windows 2000 Server. It stores information about objects on a network and makes this information available to users and network administrators through a protocol such as LDAP. On CLARiiON systems, a physical storage unit as exported from the storage array. All other volume types are created from disk volumes. A protocol for sending SCSI packets over TCP/IP networks. An iscsi endpoint, identified by a unique iscsi-recognized name that begins an iscsi session by issuing a command to the other endpoint (the iscsi target). An iscsi endpoint, identified by a unique, iscsi-recognized name that executes commands issued by the iscsi initiator. Redundant array of independent disks. A method for storing information where the data is stored on multiple disk drives to increase performance and storage capacities and to provide redundancy and fault tolerance. RAID method that provides data integrity by mirroring (copying) data onto another disk. This RAID type provides the greatest assurance of data integrity at the greatest cost in disk space. Data is striped across disks in large stripes. Parity information is stored so data can be reconstructed if needed. One disk can fail without data loss. Performance is good for reads, but slower for writes. Data is stripped, then Mirrored. Typically this is create using 4 drives minimum, in the EMC CLARiiON this can be created with only 2 drives and later additional drives can be added for additional space. Raid 1_0 when using 4 disks or more allows for better disk performance and fault tolerance. The CLARiiON storage-system term for a disk group. Storage processor on a CLARiiON storage system. On a CLARiiON EMC Solutions for Exchange 2003 CX Series iscsi 5

5 SP A SP B VSS storage system, a circuit board with memory modules and control logic that manages the storage-system I/O between the host s Fibre Channel adapter and the disk modules. Storage processor A. A generic term for the first storage processor in a CLARiiON storage system. Storage processor B. A generic term for the second storage processor in a CLARiiON storage system. (Volume Shadow Copy Service): A Windows service and architecture that coordinates various components to create consistent point-in-time copies of data called shadow copies. General environment recommendations Listed below are the best practice recommendations for Networking, Operating Systems, Clustering (MSCS and Exchange) while using iscsi for connectivity to the CLARiiON CX3-20c storage arrays. Exchange Best Practices Splitting Exchange Storage group database and log files It is highly recommended to insure that Database Files and Log files from the SAME Exchange Storage Group NOT share the same physical spindles. This prevents the possibility of loss of by multiple drives loss used an Exchange Database or Entire Storage Group. Running Exchange Best Practices Analyzer after install and every month It is recommended to run the Microsoft Exchange Best Practices Analyzer against the Exchange servers upon completion of install and follow ALL recommendations. Additionally EXBPA can be scheduled to run at intervals or through Microsoft MOM to insure the servers are running the latest definitions recommendations. Setting msexcheseparamlogbuffers = 9000 Increase log buffers to 9000 for ALL Storage Groups to avoid Log Stalls. Setting msexcheseparamcheckpointdepthmax = 5MB Decrease the Check Point Depth Max to 5MB as per the article in the link below. Decreasing this parameter on Clustered Exchange servers will allow for better failover performance. However, disk performance must be monitored to insure the increase in IO does not cause thrashing. Clustering Exchange servers EMC recommends the use of Microsoft Cluster Service (MSCS) with Exchange servers to increase the fault tolerance of hardware running Exchange server. Additionally, MSCS allows for less downtime due to Hotfix and Patch management. Training of customer administrative staff is highly recommended due to the differences in management and updating of Exchange clustered servers. Using two DC/GC per AD site Exchange relies heavily upon the Active Directory for DSAccess using the DC/GCs; it is highly recommended to have at minimum 2 Active Directory DC/GC servers per site for fault tolerance. Best Practices for the number of DC/GC rule of thumb is a 4:1 physical processor ratio where for every 4 EMC Solutions for Exchange 2003 CX Series iscsi 6

6 Exchange server physical processors there should be 1 DC/GC processor with respect to insure at minimum of 2 DC/GC servers per site for fault tolerance. Networking Best Practices Using 1Gb (GigE) switches with VLAN capabilities The use of GigE switches which are capable of setting up Virtual LANs(VLAN) to segment Production and iscsi traffic should be used for best performance. Using dedicated VLANs for production and iscsi subnets when using switches NOT explicitly used for specific functions If possible the use of dedicated switches for Production and iscsi traffic is recommended. If this is not possible insure the switches are capable of creating VLANs for Production and iscsi traffic. Using 1Gb (GigE) Network Interface Cards on the Operating System for Production and iscsi traffic on Hosts The use of independent GigE NICs dedicated for Production and dedicated for iscsi traffic should be used for best performance. One NIC for Production, One or More NICs for iscsi. CAT6 Cables for GigE connectivity The use of CAT 6 Cables has shown a dramatic results in no issue of 1000Mb connectivity which has been seen in Cat5E cables. To insure the best performance and reliability, we recommend CAT6 Cables. Network Speed and Duplexing: Auto/Auto for GigE Network and Switch ports. Insure that ALL Ports on the Switches are set to AUTO Negotiate 1000, and that the corresponding NIC cards are set to match. AUTO/AUTO is not intended by NIC manufacturers as a finished setting and is to be manually hard coded once networking configurations and speed are set. Jumbo Frame Support set to 9000 Using NIC cards which have Jumbo Frame support it is recommend to use Jumbo Frame. With the flare release.24 Jumbo Frames are supported, setting this to 9000(or setting in this range per NIC vendor). Insure the SWITCH also has this setting matched in size enabled either globally or per port depending on the vendor. Operating System Best Practices Using iscsi Initiator For best performance it is recommended to use the latest Microsoft iscsi initiator. At the time of this document, was the latest released Microsoft iscsi initiator. Removing the client for Microsoft Networks and File and Print Sharing for Microsoft Networks on iscsi NICS ONLY For best performance it is recommended that you use the latest Microsoft iscsi initiator. At the time of this document, was the latest released Microsoft iscsi initiator. Installing the latest NIC driver For best performance it is recommended to install the latest Vendor NIC driver. EMC Solutions for Exchange 2003 CX Series iscsi 7

7 iscsi Configuration. Select in the iscsi initiator: Automatically Restore this connection when system Boots Enable Multi-Path Select Advanced Choose under Local Adapter: Microsoft iscsi Initiator Source IP: Select the iscsi NIC card designated for the Spa or Spb. Target Portal: This is the iscsi Port on the CLARiiON (Spa or Spb) A0-A4, B0-B4. EMC Solutions for Exchange 2003 CX Series iscsi 8

8 Interconnect SP setup: Insure that a SINGLE NIC does not have multiple connections to the SAME SP. NIC0 to A0 and B3, NIC 1 to A3 and B0. EMC Solutions for Exchange 2003 CX Series iscsi 9

9 TCP/IP Registry Change 1 Listed below are recommendations for optimal iscsi performance. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\KeepAliveTime = Dword: ( decimal ) TCP/IP Registry Change 2 Listed below are recommendations for optimal iscsi performance. Locate the Network cards used for iscsi (note: the IP addresses will be listed under Interfaces {guid}\ipaddress key. Once you have located the IP addresses for iscsi add the TcpAckFrequency = 1 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\{A8EFA D93-95C3-4E98-AE5D-CE0E6185CA19} Using High Performance Disks for Best Exchange Performance (RPM) It is recommended with Exchange 2003 to use High Performance Disks to insure the best user experience. EMC Solutions for Exchange 2003 CX Series iscsi 10

10 Planning for performance NOT capacity The most common error people make when planning an Exchange server is designing for capacity and not for performance or IOPS (I/O per Second). The most important single storage parameter for performance is disk latency. High disk latency is synonymous with slower performance. Microsoft guidelines for good performance are as follows: Average read and write latencies below 20ms ( average disk sec/read average disk sec/write ) Maximum read and write latencies below 50ms. In today s disk technology, the increase in storage capacity of a disk drive has outpaced the increase in IOPS. Therefore, the IOPS capacity is the standard to use when planning Exchange storage configurations. Microsoft recommends Read and Write latencies to not Exceed 20ms Using diskpar to align iscsi LUNs for best performance Before a windows signature is written to any iscsi LUN, it is important to align the disk partition using DISKPART. When a Windows partition is create, it is created starting at the 64 th sector. This misaligns the partition with the physical disk. The result is that I/O operation can straddle cylinder boundaries resulting in a significant reduction in performance. Performance improvements as high as 40% have been seen after partitioning a drive using diskpart and aligning the disk. An in-depth discussion of this topic can be found in the EMC Engineering White paper Using DISKPAR and DISKPART to align partitions on Windows Basic and Dynamic Disks. The following Microsoft TechNet article also covers the topic: After LUN creation is complete on the production CLARiiON system, the active MSCS node should be able to see the LUN as a raw volum. Partition the LUN using the Microsoft command line utility DISKPART insuring to create the partition using the ALIGN=64 switch. Example 1: in the following example using disk part against drive 4. C:\>diskpart Microsoft DiskPart version Copyright (C) Microsoft Corporation. On computer: JC27Q91X32 DISKPART> list disk Disk ### Status Size Free Dyn Gpt Disk 1 Online 136 GB 112 GB Disk 2 Online 267 GB 0 B Disk 3 Online 267 GB 0 B Disk 4 Online 600 GB 600 GB Disk 5 Online 600 GB 0 B Disk 6 Online 600 GB 0 B Disk 7 Online 5114 MB 0 B Disk 8 Online 5114 MB 0 B Disk 9 Online 25 GB 0 B Disk 10 Online 25 GB 0 B Disk 11 Online 300 GB 0 B Disk 12 Online 51 GB 0 B DISKPART> select disk 4 EMC Solutions for Exchange 2003 CX Series iscsi 11

11 Disk 4 is now the selected disk. DISKPART> create partition primary align=64 DiskPart succeeded in creating the specified partition. Using the Microsoft Disk Manager then Select the Drive Letter or Mount Point to be associated with the corresponding database, log file, smtp or quorum drive. After selecting this information, Format the drive NTFS at 64k Allocation Unit Size for Databases Format the drive NTFS at 64k Allocation Unit Size for Logs Format the drive NTFS at 4k Allocation Unit Size for SMTP Format the drive NTFS at 64k Allocation Unit Size for Databases Backup to Disk and Clones. Using EMC RAID 1_0 for Log files For highest Fault tolerance and best performance using EMC Raid 1_0 starting with 2 drives and Expanding when required. Using EMC RAID 1_0 for Database files Raid 1_0 is recommended for best performance and highest fault tolerance for Microsoft Exchange Databases. Using RAID 5 if RM is to be used for instant local recovery capabilities For RM Clones utilizing Raid 5 allows for clones which must the same block size to use less disks then what are required for best performance of the Exchange production databases. Changing BOOT.ini on Windows 2003 servers with more then 1GB of Physical RAM Following the Microsoft guidance, addition of the /3GB and UserVA=3030 switch to the boot.ini on systems with more then 1GB of Physical RAM allows for more usermode memory and limiting the kernel mode memory. Backup and Restore Best Practices Using Replication Manager 5.0 and VSS to implement instant local recoverability Replication Manager 5.0 with VSS allows for almost instant local clones of the Exchange Databases and Log files. Some of the Benefits of using RM5.0 to backup Exchange 2003 are as follows: 1. Quick Backup and Restore. RM 5.0 takes only a few minutes to backup or restore and Exchange Storage Group. 2. Ease of use: RM 5.0 has a simple interface in which and IT administrator can discover applications, select Exchange storage groups and execute backup and restore operations. 3. Integrated with Microsoft Volume Shadow Copy Service (VSS). TM is integrated with the Microsoft VSS architecture when running Exchange 2003 on Windows For more on the VSS Framework, and now it is used to guarantee database consistency when snapping Microsoft 2003 server applications goto : 4. Multiple Backups: Creating 2 clones locally allows for up to 2 days of restore instantly. EMC Solutions for Exchange 2003 CX Series iscsi 12

12 5. Log Truncation: RM 5.0 Full Exchange Clone will truncate the logs only after the database checksum is completed. 6. Checksum of database integrity for every backup. RM5.0 will check the Exchange databases and log file using ESEUTIL utility from Microsoft for every backup. 7. Runs Pre and Post Scripts: RM 5.0 allows for Pre and Post Job scripts to run. 8. RM Mount Host: Having a remote mount host allows for database checksums to be run off production servers therefore not impacting the users during backups. 9. RM Clones are required to be the same block size as the Source LUNs. To insure the same size LUNs are created, using the NaviCLI command lines are recommended. Scheduling RM VSS snapshots for iscsi LUNs supporting Exchange in off hours, best practice being before or after Exchange online maintenance Online Maintenance and Online Defrag are extremely important, and during the VSS procedure, Exchange is in a backup mode and OLM/OLD do not run. Scheduling OLM/OLD and VSS to run in off hours and not during the same periods insures that OLM/OLD process is not interrupted and unable to complete. If OLM/OLD is unable to complete because of scheduling conflicts with backup software, databases will begin to grow dramatically. Naming LUNs: This allows for QUICK identification. In the below example you can see quickly the NAME identifies the size and usage. C:\Program Files\EMC\Navisphere CLI>navicli -h getlun 7 Findstr "LUN Capacity(Blocks)" Name LUN 7-25GB - SG1LG LUN Capacity(Megabytes) LUN Capacity(Blocks) Using NaviCLI for Clone Creation of Raid Groups and Storage Groups First Get the BLOCK COUNT of the Database and Log file LUNs for cloning. Using the command line interface for navisphere you can quickly gather this data using the below command line. RM requires the Destination LUN to be the exact block count size as the source. For more information refer to ManCLI.txt within the NaviCLI directory after install. C:\Program Files\EMC\Navisphere CLI>navicli -h getlun 0 Findstr "LUN Capacity(Blocks)" Name LUN 0-267GB - SG1 LUN Capacity(Megabytes): LUN Capacity(Blocks): C:\Program Files\EMC\Navisphere CLI>navicli -h getlun 7 Findstr "LUN Capacity(Blocks)" Name LUN 7-25GB - SG1LG LUN Capacity(Megabytes): LUN Capacity(Blocks): Second: Create a new Raid Group for Cloning and LUNs using NaviCLI: Raid Group 200 in this example will be for the Exchange DB and LOG clones ( 2 per * 2 SG ) C:\Program Files\EMC\Navisphere CLI>navicli -h createrg 200 1_0 1_1 1_2 1_3 1_4 1_5 1_6 1_7 1_8 1_9 -pri high Creating the LUNS within the new Raid Group 200 using RAID 5 per Recommendation #1. C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap EMC Solutions for Exchange 2003 CX Series iscsi 13

13 C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap C:\Program Files\EMC\Navisphere CLI>navicli -h bind r rg 200 -sq bc -cap Increasing NTBackup Logical Disk Buffers to 64 When using NT Backup for Backup to Disk software insure the Logical Disk Buffers are increased per Microsoft Best Practices Guide. d1cda210f3cd/exchangebackupnote.doc Increasing NTBackup Logical Disk Buffers to 1024 When using NT Backup for Backup to Disk software insure the Logical Disk Buffers are increased per Microsoft Best Practices Guide. d1cda210f3cd/exchangebackupnote.doc Increasing NTBackup Max Num Tape Buffers to 16 When using NT Backup for Backup to Disk software insure the Max Num Tape Buffers are increased per Microsoft Best Practices Guide. d1cda210f3cd/exchangebackupnote.doc CLARiiON Best Practices Balanced LUNs Design The CLARiiON CX3-20c service processors run in an active/active mode thus recommending the use of EMC s PowerPath 4.6 software for the ability of seamless failover in the event of a service processor (SP) going offline. The CLARiiON CX3-20c has four GigE ports per SP, with the capability of adding 2 Fibre Channel Ports per SP. Balancing the load against the service processors is recommended by not having ALL database LUNs on a single SP in multiple SG configurations. Replication Manager Best Practices Clone LUNs Creation using NaviCLI RM requires LUNs of equal block count, but not raid technology. Using the NaviCLI (command line interface) for the creation of the clone LUNs is recommended to insure the block count is accurate. NaviCLI example: Open a command prompt ( start \ run \ CMD.exe ) Run the below command which will report back the LUN capacity in MB along with Block Count as needed for the Clone LUNs. navicli -h getlun 0 findstr LUN EMC Solutions for Exchange 2003 CX Series iscsi 14

14 Name LUN 0-267GB - SG1 LUN Capacity(Megabytes): LUN Capacity(Blocks): Upon creation of the Raid Group using the below command to create the Clone LUN in the Example below using Raid Group 100 and binding the LUN as 200 paying special attention to the sq bc cap then using the Blocks size from the above command. navicli -h bind r rg 100 -sq bc -cap RepliStor Best Practices Microsoft Exchange / MSCS It is a Microsoft requirement that if the RepliStor source (in this case, the Exchange production) server is clustered, then the RepliStor target (failover) server must also be clustered. The RepliStor architecture was validated with a single node (1+0, lone wolf) MSCS cluster at the disaster recovery site. Avoiding the MSCS Alias service Since MSCS provides an alias service, do not use the alias service provided by RepliStor (Note: this option is not available with RepliStor when using MSCS). Instead, create an Exchange group from the MSCS source cluster. Then create a second Exchange group from the target single node cluster. Assign each group the same Virtual Exchange Server Name, thereby creating the alias. Note: the Windows NT Cluster names assigned to each cluster can differ. Making RepliStor resources dependant on Exchange SA. Using Cluster administrator, modify the MSCS system attendant resource properties of the production cluster to make the RepliStor resource a dependency. This will cause RepliStor to remain online during an Exchange production node failure and to capture file updates the System Attendant has not yet completed. RepliStor can then replicate these updates prior to failing over. Matching Drive Letters or Mount Points Check configuration of drive letters and/or mount points match on active and passive nodes. Also insure that the same drive configuration matches on the server in the DR Site. Example, if SG1 databases are on Drive G: and Logs are on Drive H: for the Active Node1, the passive node and DR node drive configuration should match. Exchange Storage Group Configuration Storage group configuration for all Exchange hosts in the environment (at both production and disaster recover sites) must be identical. This applies to the number of storage groups, paths to.edb,.stm files and log files, in addition to the number and names of the mail stores and storage groups. EMC Solutions for Exchange 2003 CX Series iscsi 15

15 Appendix: A 500 User Mid-Sized Enterprise Building Block Design Storage Layout Figure (X) Initial 500 user Mid-Sized Enterprise Building Block. 1 Storage Group In this configuration, 4 Raid groups are created. Raid Group 0 for the HOTSPARE, Raid Group 1 will contain drive 0_0_0-0_0_1 with two 25GB and one 40GB LUN, Raid Group 2 will contain 0_0_2-0_0_5 will contain 1 300GB LUN, and Raid Group 3 will contain 0_0_6-0_0_9 and will contain 1 267GB LUN utilizing all available space. For Clustering a lun is required to be created at 1GB. Using space from Raid Group 1 Create one 1GB LUN for this purpose. Figure(Y) 1000 user Mid-Sized Enterprise Building Block. 2 Storage Groups In this configuration, 5 Raid groups are created. Raid Group 0 for the HOTSPARE disk 0_0_14, Raid Group 1 will contain drive 0_0_0-0_0_1 with two 25GB and one 40GB LUN, Raid Group 2 will contain 0_0_2-0_0_5 will contain 1 300GB LUN, Raid Group 3 will contain 0_0_6-0_0_9 and will contain 1 267GB LUN utilizing all available space, and Raid Group 4 will contain 0_0_10-0_0_13 and will contain 1 267GB LUN utilizing all available space. For Clustering a LUN is required to be created at 1GB. Using space from Raid Group 1, create one 1GB LUN for this purpose. EMC Solutions for Exchange 2003 CX Series iscsi 16

16 Appendix: B Connecting iscsi Initiators to CX Targets EMC Solutions for Exchange 2003 CX Series iscsi 17