The DataCore Server. Best Practice Guidelines. February The Data Infrastructure Software Company

Transcription

1 The DataCore Server February 2018 The Data Infrastructure Software Company

2 Table of contents Changes made to this document 3 Design objectives 5 The server hardware 7 BIOS 7 CPUs 8 Power 9 System memory (RAM) 9 Fibre channel and iscsi connections 12 Multi-ported vs Single-ported adaptors 12 ISCSI connections 12 RAID controllers and storage arrays 14 The operating system 16 Microsoft Windows 16 Windows system settings 16 Microsoft Windows Updates 19 Third-party software 21 Upgrading (or reinstalling) Microsoft Windows 22 Upgrading the SANsymphony software 24 TCP/IP Networking 25 The SANsymphony Server Group 25 Running the SANsymphony Management Console 27 Major SANsymphony features 30 Disk Pools 30 Replication 35 Snapshot 41 Previous Changes 43 Page 2

3 Changes made to this document The most recent version of this document is available from here: All changes since August 2017 Updated CPUs This section has been re-written and simplified as some of the terminology was confusing. Windows service packs, updates, security and hot fixes This section has been renamed and updated for Windows 2012 and 2016 with many of the explanations simplified. Disk Pools Minor updates to the text, mainly for clarity concerning existing Disk Pools vs new Disk Pools. All previous changes Please see page 40. Page 3

4 Overview Each DataCore implementation will always be unique and giving advice that applies to every installation is therefore difficult. The information in this document should be considered as a set of general guidelines and not necessarily strict rules. DataCore cannot guarantee that following these best practices will result in a perfect solution there are simply too many separate (and disparate) components that make up a complete SANsymphony installation, most of which are out of DataCore Software s control but these guidelines will significantly increase the likelihood of a more secure, stable and high-performing SANsymphony installation. This guide assumes that DataCore-specific terms - e.g. Virtual Disk, Disk Pool, CDP and Replication etc. including their respective functions will be understood. It does not replace DataCore technical training nor does it remove any need to use a DataCore Authorized Training Partner and assumes fundamental knowledge of Microsoft Windows and the SANsymphony suite of products. Which SANsymphony versions does this document apply to? This document applies to any SANsymphony 10.x version and SANsymphony-V 9.0 PSP4 Update 4 (earlier versions are now end of life). Any recommendations that are version-specific will be clearly indicated within the text. Page 4

5 Design objectives Design objectives The design objective of any SANsymphony configuration is an end-to-end process (from the user to the data) as well as availability, redundancy and performance. None of these can be considered in isolation but as integrated parts of an entire infrastructure. The information in this document provides some high level recommendations that are often overlooked or simply forgotten about. Avoid complexity The more complex the design, the more likely unforeseen problems will occur. Complex design can also make a system difficult to maintain and support especially as it grows in size. A simple approach is recommended whenever possible. Avoid single points of failure Dependencies between major components (e.g. switches, networks, fabrics and storage arrays) can impact the entire environment if they fail. Distribute each component over separate racks, separate rooms, separate buildings and even separate 'sites'. Keep storage components away from public networks. Avoid connecting redundant devices to the same power circuit. Use redundant UPS protected power sources and connect every device to it. A UPS back-up does not help much if it fails to notify a Host to shut down because that Host s management LAN switch itself was not connected to the UPS backed power circuit. Use redundant network infrastructures and protocols where the failure of one component does not make access unavailable to others. Do not forget environmental components. A failed air conditioner may collapse all redundant systems located in the same datacenter. Rooms on the same floor may be affected by a single burst water pipe (even though they are technically separated from each other). Set up monitoring and event notification Highly available systems keep services alive even if half of the environment has failed but these situations must always be recognized and responded to in a timely manner so that they can be fixed as soon as possible by the personnel responsible to avoid further problems. Documentation Document the environment properly; keeping it up-to-date and accessible. Establish 'shared knowledge' between at least two people who have been trained and are familiar with all areas of the infrastructure. Page 5

6 Design objectives User access Make sure that the difference between a 'normal' server and a DataCore Server is understood. A DataCore Server should only be operated by a trained technician. Best practices are not pre-requisites Some of the Best Practices listed here may not be applicable to your installation or cannot be applied because of the physical limitations of your installation for example, it may not be possible to install any more CPUs or perhaps add more PCIe Adaptors or maybe your network infrastructure limits the number of connections possible between DataCore Servers and so on. Therefore, each set of recommendations is accompanied by a more detailed explanation so that when there is a situation where a best practice cannot be followed, there is an understanding of what and how that may limit your SANsymphony installation. Also see: DataCore Training Overview How to become a DataCore Certified Implementation Engineer (DCIE) Page 6

7 The server hardware BIOS Many server manufacturers often have their own 'Low Latency Tuning' recommendations. DataCore always recommend follow those recommendations settings unless they contradict anything recommended below. Server manufacturers may use different names or terminology for the settings listed below. Collaborative power control This should always be disabled where possible as it overrides many of the BIOS settings listed in this section. Command per Clock (CPC Override/Mask) This should be enabled as it can increase the overall speed of memory transfer. Hemisphere Mode Hemisphere mode can improve memory cache access both to and from the memory controllers but the server hardware usually has to have memory modules of the same specification to take advantage of this setting. This should be set this to Automatic. Intel Turbo Boost Turbo Boots should be disabled - in testing DataCore discovered Intel Turbo Boost can, in extreme cases, over-clock a CPU which leads to erratic behavior that impacts SANsymphony's I/O Scheduler. Non-Uniform Memory Access (NUMA) Snoop options - Enable the Early Snoop option (if available). Cluster options - Disable all NUMA Cluster option settings. Power Management settings Power saving/c-states - Disable any C-states ('C0' State) or power saving settings. SANsymphony relies on the CPUs being available all the time and not have to wait while the CPU is 'woken up' from an idle state. Static High If available, set the power management settings to static high to disable any additional CPU power saving. Page 7

8 Hardware Server hardware CPUs All x64 processors (except for Intel s Itanium family) is supported for use in a DataCore Server. DataCore recommend using server-class CPUs than those intended for workstation' use. The speed of CPUs DataCore Software has not found any significant performance differences between CPU manufacturers for processors that share similar architectures and frequencies. Even so, faster (i.e. higher frequency) CPUs are always preferred over slower ones as they can process more instructions per second. DataCore also prefer using the fastest cores when possible rather than more-but-slower cores. Please consult your server vendor to see if any additional CPU Sockets are necessary to be able to use all of the available PCIe/Memory- Sockets on the server s motherboard. Hyper-Threading (Intel) For Intel CPUs manufactured after 2014, DataCore recommend that Hyper-Threading is enabled as testing has shown this can help increase the number of threads for SANsymphony's I/O Scheduler, allowing more I/O to be processed at once. For earlier Intel CPUs DataCore recommend that Hyper-Threading be disabled as testing has shown that older CPUs would behave erratically with Hyper-threading enabled. What is the recommended number of CPUs in a DataCore Server? At least 4 logical processors for the base SANsymphony Software. Note: You can install the software using just 2 logical processors but this is minimum requirement. 1 additional logical processor for each fibre channel port 3 additional logical processors for each pair of iscsi ports. Software ISCSI connections in general have a much larger overhead compared to fibre channel ports, mainly due to the extra work required encapsulating/de-encapsulating of SCSI data to/from IP packets. 1 additional logical processor for both the Live and Performance Recording features. 2 additional logical processors for the (Asynchronous) Replication feature. Hyperconverged Virtual SAN running SANsymphony alongside Hyper-V (Windows only) Only the Hosts are running in Virtual Machines, the SANsymphony software is still installed on a 'physical' server. Follow the recommendations on the previous page but consider that additional logical processors may be needed to manage the Hyper-V software and Host VMs. Please consult Microsoft s Hyper-V documentation for more information. Hyperconverged Virtual SAN running SANsymphony inside a Virtual Machine The DataCore Servers and Hosts are all running in Virtual Machines. In this case, treat 'logical processor' requirements as 'virtual processor' (vcpu) requirements. Consideration must be taken though with regard to the allocation of resources to all other Hosts and how the Page 8

9 Hardware Server hardware resources of the Hypervisors 'physical' CPUs are shared between Host VMs and the VMs used for DataCore Servers. Even if the same number of vcpus were to be created to match those of an equivalent physical DataCore Server, there is still no guarantee that all of these vcpus would be used at the same rate and throughput as physical CPUs. Also see: DataCore Hyper-converged Virtual SAN Best Practices Guide Power Use redundant and uninterruptable power supplies (UPS) whenever possible. Also see: UPS Support System memory (RAM) The amount of system memory The minimum amount of memory for a DataCore Server is 8GB. The recommended amount of memory for a DataCore Server depends entirely on the size and complexity of the SANsymphony configuration. Use the DataCore Server Memory Considerations document available from the Support Website to calculate the memory requirement for the type, size and complexity of the SANsymphony configuration, always allow for plans of future growth. Also see: DataCore Server Memory Considerations The type of system memory To avoid any data integrity issues while I/O is being handled by the DataCore Server s own Cache, ECC Memory Modules should be used. If a Server CPU's uses NUMA architecture then all the physical memory modules should have the same specification. See the NUMA Group Size Optimization and Hemisphere Mode entries from the BIOS section on page 7 for more information. Page 9

10 Hardware Server hardware A summary of server hardware recommendations BIOS Collaborative power control should be disabled. CPC Override/Mask should be enabled. Hemisphere Mode should be set to Automatic. Intel Turbo Boost should be disabled. NUMA Group Size Optimization/Node Interleaving should be enabled and set to Flat (if the option is available). Power saving (C-states) should all be disabled but Static High should be enabled. CPU Generally Use server class processors. Use less-but-faster cores rather than more-but-slower cores Enable Hyper-Threading (Intel) on CPUs from 2014 or newer. Disable Hyper-Threading (Intel) on CPUs older than Physical DataCore Servers At least 4 logical processors for the base SANsymphony Software. Note: You can install the software using just 2 logical processors but this is minimum requirement. 1 additional logical processor for each fibre channel port 3 additional logical processors for each pair of iscsi ports. Software ISCSI connections in general have a much larger overhead compared to fibre channel ports, mainly due to the extra work required encapsulating/de-encapsulating of SCSI data to/from IP packets. 1 additional logical processor for both the Live and Performance Recording features. 2 additional logical processors for the (Asynchronous) Replication feature. Hyperconverged DataCore Servers Running SANsymphony alongside Hyper-V (Windows only) o Follow the recommendations as for Physical DataCore Servers with additional logical processors required for the Hyper-V software. Running SANsymphony inside a Virtual Machine o Follow the recommendations as for Physical DataCore Servers (substitute 'logical' with 'virtual' processors) consideration must be taken for all other 'Host' VMs so that the DataCore Server VM and Host VM's are not competing for resources. Page 10

11 Hardware Server hardware Power Use redundant power supplies. Use an uninterruptable power supply (UPS). System Memory See DataCore Server Memory Considerations from the Support Website: Use ECC Memory. Enable CPC Settings. Page 11

12 Hardware Fibre channel and iscsi connections Fibre channel and iscsi connections Multi-ported vs Single-ported adaptors Fibre Channel host bus adapters (HBA) and network interface cards (NICs) are often available in single, dual and quad-port configurations but often there can be very little difference in performance capabilities when comparing two single-ported adaptors with one dual-port adaptor, or two dual-port HBAs with one quad-port HBA [1]. There is however a significant implication for high availability when using a single adaptor - even if it has multiple ports in it - as most types of adaptor failures will usually affect all ports on it rather than just one (or some) of them. Using many adaptors that have a small number of ports on them will reduce the risk of multiple port failures happening at the same time. ISCSI connections Link aggregation and SCSI Load Balancing For better performance use faster, separate network adaptors and links instead of teaming multiple, slower adaptors. For high availability use more, individual network connections and multipath I/O software than rely on either teaming or spanning tree protocols to manage redundancy. Just like Fibre Channel environments, that use independent switches for redundant fabrics, this also prevents network-loops making spanning tree protocols obsolete and simplifies the overall iscsi implementation. Fundamentally, SCSI load-balancing and failover functions are managed by Multipath I/O protocols [2] ; TCP/IP uses a completely different set of protocols for its own load-balancing and failover functions. When SCSI commands, managed by Multipath I/O protocols but carried by TCP/IP protocols are combined (i.e. iscsi), then interaction between the two protocols for the same function can lead to unexpected disconnections or even complete connection loss. NIC teaming NIC teaming is not recommended for iscsi connections as it adds more complexity (without any real gain in performance); and although teaming iscsi Targets - i.e. Front-end or Mirror ports - would increase the available bandwidth to that target, it still only allows a single target I/O 1 This assumes that there are always an adequate number of PCIe Lanes available in the PCI Slot being used for the adapter. Please refer to your server hardware vendor s own documentation for this. 2 Mirrored Virtual Disks that are configured to use multiple iscsi Mirror paths on the DataCore Server are, by default, autoconfigured to be managed by Microsoft s MPIO using the Round Robin with Subset Policy. Page 12

13 Hardware Fibre channel and iscsi connections queue rather than, for example, two, separate NICs which would allow two, independent target queues with the same overall bandwidth. Spanning Tree Protocols (STP/RSTP/MSTP) None of the Spanning Tree Protocols - STP, RSTP or MSTP are recommended on networks used for iscsi as they may cause unnecessary interruptions to I/O; for example, other, unrelated devices generating unexpected network-topology causing STP to re-route iscsi commands inappropriately or even blocking them completely from their intended target. Also see: The DataCore and Host Servers - Qualified Hardware Known Issues - Third Party Hardware and Software A summary of recommendations for Fibre channel and iscsi connections Fibre channel and ISCSI connections Use many adapters with a smaller numbers of ports on them as opposed to fewer adapters with larger numbers of ports. ISCSI connections Use faster, separate network adaptors instead of NIC teaming. Do not use NIC teaming or STP protocols with iscsi connections. Use more, individual network connections (with Multipath I/O software) to manage redundancy. Use independent network switches for redundant iscsi networks. Page 13

14 Hardware RAID controllers and storage arrays The DataCore Server's Boot disk Configure the DataCore Server s own Boot Disk for redundancy; use RAID 1 (usually simpler, less complex to configure and has less overhead than RAID 5). Disks used for DataCore Disk Pools RAID and Storage Array controllers used to manage physical disks in Disk Pools need to be able to handle I/O from multiple Hosts connected to the DataCore Server; so for high performance/low latency hardware, use independent buses or backplanes designed for heavy workloads ( workstation hardware is not usually designed for such workloads). A low-end RAID controller will deliver low-end performance. An integrated (onboard) RAID controller that is often supplied with the DataCore Server may only be sufficient to handle just the I/O expected for the boot drive. Controllers that have their own dedicated CPU and cache are capable of managing much higher I/O workloads and many more physical disks. Consult with your storage vendor about the appropriate controller to meet your expected demands. If both 'fast' and 'slow' disk types share the same disk controller in the storage array (e.g. an SSD sharing the same disk controller as a SAS RAID5 set), then the slower disks on that controller can hold up I/O to the faster disks. DataCore recommend is to have a separate disk controller for each differential of disk speed type. If there is no choice but to mix different disk speed types on the same disk controller - for example mixing SSD with SAS - then in this example make sure the SAS disk have 'no RAID' (or RAID0) configured and use SANsymphony's Disk Pool mirroring feature as this should be faster than hardware RAID mirroring. Also see: Storage Hardware Guideline for use with DataCore Servers The section 'Mirroring Disk Pools' from: The DataCore and Host Servers - Qualified Hardware Known Issues - Third Party Hardware and Software Page 14

15 Hardware RAID controllers and storage arrays A summary of recommendations for RAID controllers and storage arrays RAID controllers and Storage Arrays Use high performance/low latency hardware with independent buses or backplanes designed for heavy workloads ( workstation hardware is not usually designed for such workloads). Avoid using onboard disk controllers for anything other than the DataCore Servers own Boot Disks. Configure the DataCore Server s Boot Disk for RAID1. Use a separate controller for physical disks used in either Disk Pools or for the Replication buffer. If mixing fast and slow disks on the same controller then use Disk Pool mirroring instead of hardware RAID on the slower disks. Also see: Storage Array Guidelines for DataCore Servers from the Support Website: Page 15

16 The operating system Microsoft Windows On Server Vendor versions of Windows Use non-oem versions of Microsoft Windows to avoid the installation of unnecessary, thirdparty services that will require extra system resources and so potentially interfere with SANsymphony. 'R2' editions of Windows Only install R2 versions of Windows that are listed either in the 'Configuration notes' section of the SANsymphony release notes that is currently installed or in the 'SANsymphony Component Software Requirements' section here: Windows system settings Synchronize all DataCore Server system clocks with each other and connected Hosts While the system clock has no influence on I/O - from Hosts or between DataCore servers there are some operations that are, potentially, time-sensitive. SANsymphony Console Tasks that use a Scheduled Time trigger. Also see: Automated Tasks Continuous Data Protection s retention time settings. Also see: Continuous Data Protection (CDP) 'Temporary' license keys e.g. license keys that contain a fixed, expiration date (e.g. for trial, evaluation or migration purposes). 'Significant' differences between DataCore Server system clocks can generate unnecessary 'Out-of-synch Configuration' warnings after reboot of a DataCore Server in a Server Group Also see: Out-of-synch Configurations It is also recommended to synchronize all of the host s system clocks as well as any SAN or Network switch hardware clocks (if applicable) with the DataCore Servers. This can be especially helpful when using DataCore s VSS on a host but also generally to help with any troubleshooting where a host s own system logs need to be checked against those of a Page 16

17 The operating system DataCore Server. Many SAN events often occur over very short periods of time (e.g. Fibre Channel or ISCSI disconnect and reconnection issues between Hosts and DataCore Servers). Power Options Select the High Performance power plan under Control Panel\Hardware\Power Options. Remember some power options are also controlled directly from within the server BIOS; see the section on page 7. Startup and Recovery/System Failure The SANsymphony installer will by default enable DataCore-preferred settings automatically. No additional configuration is required. Virtual Memory/Page File SANsymphony does not use the page file for any of its critical operations. No additional settings are required and the default page file settings should be used. The default size of the page file created by Windows is determined by the amount of Physical Memory installed in the DataCore Server along with the type of memory dump that is configured. The SANsymphony installer will change the memory dump type to Kernel Memory Dump ensure that if any crash analysis is required from the DataCore Server that the correct type of dump file is generated. A DataCore Server that has a small boot disk and large amounts of physical memory may end up with a Page File that fills the boot disk after the installation. In this case, it is still recommended to keep the Kernel Memory Dump setting but manually enter a custom value for the page file size as large as is practically possible (for your boot disk) by unchecking the Automatically manage paging file size for all drives option. Page 17

18 The operating system Enable user-mode dumps User-mode dumps are especially useful to help analyze problems that occur for any SANsymphony Management or Windows Console issues (i.e. non-critical, Graphical User Interfaces) if they occur. 1. Open regedit and browse to: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting 2. Add a new 'Key' underneath 'Windows Error Reporting' and rename it 'LocalDumps' 3. Add a new REG_DWORD with a Name of 'DumpType' and a Data value of '2' 4. Close 'regedit' to save the changes. Also see: Collecting User-Mode Dumps Page 18

19 The operating system Microsoft Windows Updates Automatic or Manual updates? Do not configure the DataCore Server to apply updates automatically as this will usually result in the DataCore Server rebooting in an 'unmanaged' way, for example; without first making sure that the SANsymphony software has stopped properly, or if other DataCore Servers in the Server Group are still running. Always apply windows updates manually. How to configure Windows Update settings: 1. Open an administrative cmd or PowerShell window and run the command 'sconfig'. Locate the current 'Windows Update Settings': 2. If the current setting is 'Automatic' then enter option '5' and then when prompted, enter 'M' (for manual) and press enter: 3. Wait for the setting to be applied and when prompted: 4. Click 'OK' and verify that the Windows Update Setting value is now 'Manual'. 5. Finally, enter '15' to exit out of the 'sconfig' utility and close the window. Page 19

20 The operating system Windows service packs Only install services packs that are listed either in the 'Configuration notes' section of the SANsymphony release notes that is currently installed or in the 'SANsymphony Component Software Requirements' section here: Windows security updates Please always refer to the SANsymphony software release notes in case there are any updates that are known to cause issues or need additional consideration before installing. Generally though, DataCore recommend that you always apply the latest security updates as they become available. Windows software updates - including 'Quality' and 'Monthly' rollups Please always refer to the SANsymphony software release notes in case there are any updates that are known to cause issues or need additional consideration before installing. Generally though, DataCore recommend that you always apply the latest software updates as they become available. Windows software updates 'Preview' rollups DataCore recommend that you do not apply 'Preview' rollups unless specifically asked to do so by Technical Support. Windows software updates third party drivers DataCore recommend that you do not apply third-party driver distributed by Windows updates (e.g. Fibre Channel drivers). If you are in any doubts please contact Technical Support for advice. Windows Hotfixes Occasionally, Microsoft will make hotfixes available before they are distributed vai normal Windows Update. If a hotfix is not listed in the 'SANsymphony Component Software Requirements' section here: AND it is not being distributed as parts of a normal Windows software update then do not apply it. If you are in any doubts please contact Technical Support for advice. Page 20

21 The operating system Third-party software It is recommended not to install third-party software on a DataCore Server. SANsymphony requires significant amounts of system memory as well as CPU processing; it will also prevent certain system devices (e.g. Disk devices) from being accessed by other software components that may be installed on the DataCore Server which may lead to unexpected errors from those other software components. The purpose of the DataCore Server should not be forgotten and trying to run the DataCore Server as a Domain Controller or as a Mail Server/Relay for example, as well as SANsymphony, must not be done as this will affect the overall performance and stability of the DataCore Server. DataCore recognize that certain types of third-party software are required to be able to integrate the DataCore Server onto the user s network. These include: Virus scanning applications UPS software agents The server vendor s own preferred hardware and software monitoring agents In these few cases, and as long as these applications or agents do not need exclusive access to components that SANsymphony needs to function correctly (i.e. Disk, Fibre Channel or iscsi devices), then it is possible to run these alongside SANsymphony. Always consult the third-party software vendor for any additional memory requirements their products may require and refer to the Known Issues - Third-party Hardware and Software document for any potential problems with certain types of third-party software that have already been found to cause issues or need additional configuration. DataCore Support may ask for third-party products to be removed in order to assist with Troubleshooting. Also see: Changing Cache Size DataCore Server Memory Considerations Known Issues - Third-party Hardware and Software Qualified Software Components Page 21

22 The operating system Upgrading (or reinstalling) Microsoft Windows Versions of Windows that have passed qualification for a specific version of SANsymphony will be listed in both the SANsymphony Software release notes and the SANsymphony minimum requirements page. Never upgrade in-place to a newer version of Windows operating system, for example upgrading from Windows 2008 to Windows 2012 or upgrading from Windows 2012 to Windows 2012 R2; even if the newer version is considered qualified by DataCore the upgrade will stop the existing SANsymphony installation from running. Instead of an in-place upgrade the DataCore Server s operating system must be installed as new. R2 versions of a particular Windows Operating System also need to be qualified for use on a DataCore Server. Any R2 versions of Windows that have passed qualification for a specific version of SANsymphony will be listed in both the SANsymphony Software release notes and the SANsymphony minimum requirements page. Also see: How to reinstall or upgrade the DataCore Server's Windows Operating System SANsymphony s minimum requirements Software Downloads and Documentation - SANsymphony release notes Page 22

23 The operating system A summary of recommendations for the operating system The Windows operating system Use non-oem versions of Microsoft Windows. Only use 'R2' versions of Windows that are listed in the Release Notes or on the Prerequisites page from the DataCore Website. Windows system settings Synchronize the system clocks between all DataCore Servers in the same Server Group, any associated Replication Groups and all Hosts. Use the High Performance power plan. Enable user-mode dumps for any errors that may occur when using the SANsymphony Management Console. Use the default Page File settings. Windows service packs, security updates, software updates and rollups Do not configure the DataCore Server to apply updates automatically, always apply windows updates manually. Always apply the latest security, software updates and/or monthly rollups updates as they become available. Do not apply preview rollups. Do not apply hot fixes that are not included as part of a normal Windows Update Service and are not listed in the SANsymphony minimum requirements web page. Do not apply any Windows updates for (third-party) drivers without first contacting DataCore Technical Support. Installing third-party software on the DataCore Server It is recommended not to install third-party software on a DataCore Server. Upgrading (or reinstalling) the Windows Operating System Only use versions of the Windows operating system that have been qualified. Never upgrade in place newer versions of Windows operating system. Only use R2 versions of Windows that have been qualified. Never upgrade in place versions of Windows to its R2 equivalent. Page 23

24 Upgrading the SANsymphony software All the instructions and considerations for updating existing versions of SANsymphony or when upgrading to a newer, major version of SANsymphony are documented in the SANsymphony Software release notes. The SANsymphony Software release notes are available either as a separate download or come bundled with the SANsymphony software. Also see: Software Downloads and Documentation - SANsymphony release notes Page 24

25 TCP/IP Networking SANsymphony s Console, the VMware vcenter Integration component Replication and Performance Recording function (when using a remote SQL Server) all use their own separate TCP/IP session. To avoid unnecessary network congestion and delay as well as losing more than one of these functions at once should any problems occur with one or more network interfaces, we recommend using a separate network connection for each function. The SANsymphony Server Group The Controller node Where a Server Group has two or more DataCore Servers in it, one of them will be designated as the controller node for the whole group. The controller node is responsible for managing what is displayed in the SANsymphony Console for all DataCore Servers in the Server Group for example; receiving status updates for the different objects in the configuration for those other DataCore Servers (e.g. Disk Pools, Virtual Disks and Ports etc.), including the posting of any Event messages for those same objects within the SANsymphony console. The controller node is also responsible for the management and propagation of any configuration changes made in the SANsymphony Console regardless of which DataCore Server s configuration is being modified, and makes sure that all other DataCore Servers in the Server Group always have the most recent and up-to-date changes. The election of which DataCore Server is to become the controller node is decided by the SANsymphony software automatically and whenever; A DataCore Server is removed from or added to the existing Server Group The existing controller node is shutdown The existing controller node becomes unreachable via the TCP/IP network to the rest of the Server Group (e.g. an IP Network outage). The decision on which DataCore Server becomes the controller node is decided automatically between all the Servers in the Group and cannot be manually configured. It is also important to understand that the controller node does not manage any Host, Mirror or Back-end I/O (i.e. in-band connections) for other DataCore Servers in the Server Group. In-band I/O is handled by each DataCore Server independently of the other Servers in the Server Group, regardless if it is the elected controller or not. Nor does it send or receive Replication data Page 25

26 TCP/IP Networking configured for another DataCore Server in the same Server Group, although it will manage all Replication configuration changes and Replication status updates regardless if it is the Source Replication Server or not. The Connection Interface setting Except for iscsi I/O, all other TCP/IP traffic sent and received by a DataCore Server is managed by the SANsymphony Connection Interface setting. This includes: When applying SANsymphony configuration updates to all servers in the same Server Group. Any UI updates while viewing the SANsymphony Console, including state changes and updates for all the different objects within the configuration (e.g. Disk Pools, Virtual Disks, Snapshots and Ports etc.). Configuration updates and state information to and from remote Replication Groups Configuration updates when using SANsymphony s VMware vcenter Integration component. SQL updates when using a remote SQL server for Performance Recording The Connection Interface s default setting ( All ) means that SANsymphony will use any available network interface on the DataCore Server for its host name resolution, this is determined by the Windows operating system and how it has been configured and connected to the existing network. It is possible to change this setting, and choose an explicit network interface (i.e. IP Address) to use for host name resolution instead, but this requires that the appropriate network connections and routing tables have been set up correctly and are in place. SANsymphony will not automatically retry other network connections if it cannot resolve to a hostname using an explicit interface. We recommend leaving the setting to All and use the appropriate Hosts file or DNS settings to control host name resolution. Page 26

27 TCP/IP Networking Running the SANsymphony Management Console On the DataCore Server Even if the SANsymphony Console is used to log in to one of the other DataCore Servers in the group (i.e. an unelected node ) that other server will still connect directly to the controller node to make configuration changes or to display the information in its own SANsymphony Console. This means that all DataCore Servers in the same Server Group must have a routable TCP/IP connection to each other so that if the controller node moves to a different server, then the new controller node must also be able connect to all of the remaining DataCore Servers in the group [1]. On a Workstation Workstations which only have the SANsymphony Console component installed cannot become controller nodes and never directly send or receive configuration information for any Server Group they connect to. Just like an unelected node the workstation will connect to the controller node to make configuration changes or to display the information in its own SANsymphony Console (See Understanding the Controller Node concept on the previous page). This means that even if the workstation is on a separate network segment from the DataCore Servers (e.g. in a different vlan) it must still be able to send and receive TCP/IP traffic to and from all the DataCore Servers in that vlan. Other IP-based protocols While it is technically possible to share ISCSI I/O, Replication data and the SANsymphony Console s own inter-node traffic over the same TCP/IP connection, for performance as well as losing more than one of these functions at once, we recommend using dedicated and separate network interfaces for each iscsi port. NIC teaming and Spanning Tree Protocols (STP/RSTP/MSTP) The DataCore Server 'management' connection For all TCP/IP traffic where Multipath I/O protocols are not being used (i.e. non-iscsi traffic), we recommend to use NIC teaming to provide redundant network paths to other DataCore Servers. We also recommend that each NIC that is teamed is in its own separate network and that failover mode is used rather than load balancing as there is no specific performance 1 Re-election of the controller node takes place if the node is shutdown or if it becomes unavailable on the network to the rest of the Server Group for any reason. Page 27

28 TCP/IP Networking requirement for Inter-node communication as the TCP/IP and using fail over mode means that configuring and managing the network connections and switches is simpler. It also makes troubleshooting any future connection problems easier as well. ISCSI connections See page 12 Replication See page 35 Windows Hosts file / DNS settings There is no preference for managing DataCore Server host name resolution between using the local Hosts file or DNS. Either method can be used. DataCore do recommend however using Host Name resolution over just using IP addresses as it is easier to manage any IP address changes that might occur, planned or unexpected, by being able to simply update any Hosts file or DNS entries instead of reconfiguring a Replication group or remote SQL server connection for Performance Recording (i.e. manually disconnecting and reconnecting), which is disruptive. When using a Hosts file, do not add any entries for the local DataCore Server but only for the remote DataCore Servers and do not add multiple, different entries for the same server (e.g. each entry has a different IP address and/or server name for the same server) as this will cause problems when trying to (re)establish network connections. Firewalls and network security The SANsymphony software installer will automatically reconfigure the DataCore Server s Windows firewall settings to allow it to be able to communicate with other DataCore Servers in the same Server or Replication groups. No additional action should be required. If using an external firewall solution or another method to secure the IP networks between servers then refer to the Windows Security Settings Disclosure for the full list of TCP Ports required by the DataCore Server and ensure that connections are allowed through. Also see: Windows Security Settings Disclosure Page 28

29 TCP/IP Networking A summary of recommendations for TCP/IP Networking ISCSI connections See ISCSI Hardware on page on page 12 for more information iscsi. Other required TCP/IP connections Leave the DataCore Server s Connection Interface setting as default ( ALL ). Use either Hosts or DNS settings to control all host name resolution for the DataCore Server. Use a managed Hosts file (or DNS) instead of just using IP addresses. Never install a Windows service pack that has not been qualified. Any Windows updates and security fixes that are currently available from Microsoft s Windows Update Service should be applied whenever possible. For Firewalls and other network security requirements please refer to Windows Security Settings Disclosure via the online help: Webhelp/windows_security_settings_disclosure.htm TCP/IP Network Topology Ensure DataCore Servers in the same Server Group all have routable TCP/IP connections to each other at all times. Ensure Workstations that are only using the SANsymphony Console has a routable TCP/IP connection to all DataCore Servers in the Server Group at all times. If using SANsymphony s VMware vcenter Integration component, ensure the server running the vcenter has a routable TCP/IP connection to all DataCore Servers in the Server Group at all times. Do not route non-iscsi TCP/IP traffic over iscsi connections. Use dedicated and separate TCP/IP connections for each of the following: The SANsymphony Console Replication transfer Performance Recording when connecting to a remote SQL server. Use NIC teaming, in failover mode, for inter-node connection redundancy for non- ISCSI TCP/IP traffic with separate networks for each NIC. Use independent network switches for redundant TCP/IP networks used for inter-node communication between DataCore Servers. Page 29

30 Major SANsymphony features Disk Pools Please note that the best practices for Disk Pools that are documented here may not always be practical (or possible) to apply to existing Disk Pools in your configuration, but nevertheless should be considered for all new Disk Pools. These recommendations are also for optimal performance not minimal capacity where a larger storage allocation unit (SAU) means less additional work that the Disk Pool has to do to keep its own internal indexes up-to-date, which results in better overall performance within the Disk Pool, especially for very large configurations. While a larger SAU size will mean that there is often more, initial capacity allocated by the Disk Pool, the chance of newer Host writes needing to allocate yet more new SAUs will be less likely and will instead be written to one of the already allocated SAUs. The following applies to all types of Disk Pools, including normal, shared, SMPA and Bulk Pools. Also see: The Disk Pool Catalog The Disk Pool Catalog is an index that is used to manage information about each Storage Allocation Unit's location, its allocation state and its relationship to the storage source it is allocated for within the Disk Pool. The Catalog is stored on one or more of the physical disks in the Disk Pool - also used for SAU data - but in a location that only the Disk Pool driver can access. Each DataCore Server's Disk Pool has its own, independent Catalog regardless if the Disk Pool is shared or not. Information held in the Catalog includes: Whether an SAU is allocated to a Storage Source or not The Physical Disk within the Disk Pool that an allocated SAU can be located The Storage Source an allocated SAU 'belongs' to The Virtual Disk's Logical Block Address (LBA) that the allocated SAU represents when accessed by a Host. Whenever an SAU is allocated, reclaimed or moved to a new physical disk within the same Disk Pool, the Catalog is updated. It is important that Catalog updates happen as fast as possible to not to interfere with other I/O within the Disk Pool; for example, if the Catalog is being updated for one SAU allocation and another Catalog update for a different SAU is required, then this other Catalog update will have Page 30

31 Disk Pools to wait for a short time before its own index can be updated. This can be noticeable when a lot of SAUs need to be allocated all within a very short time; and while the Disk Pool will try to be as efficient as possible when handling multiple updates for multiple SAU, there is an additional overhead while the Catalog is updated for each new allocation before the I/O written to the SAU is considered complete. This can, in extreme cases, result in unexpected I/O latency during periods of significant SAU allocations. Therefore we recommend that the Catalog be located on the fastest disk possible within the Disk Pool. Where is the Catalog located? The Catalog is always stored within 1GB of the start of a physical disk's LBA space and always on the first physical disk added to the Disk Pool. If a second physical disk is added to the Disk Pool, then a backup copy of the Catalog is stored on that second physical disk the Catalog on the first physical disk is then considered as the primary copy. Tier assignment of the physical disk has no influence on where the Catalog will be stored. Any further physical disks added to the Disk Pool will not be used to store further copies. If the physical disk that holds the primary Catalog is mirrored within the Disk Pool, then the physical disk used as the mirror will now hold the backup Catalog. If the backup Catalog was previously on another physical disk in the Disk Pool before the mirroring took place, then then this other (non-mirrored) physical disk will no longer hold the backup Catalog. There is only ever a maximum of two copies of the Catalog in a Disk Pool at any time. Also see: It is important that any Catalog update can happen as fast as possible to not interfere with other I/O within same the Disk Pool, as in extreme cases where a lot of sudden, intensive I/O from a Host requires a lot of Catalog updates, can end up causing increased I/O wait times for other SAU read and write requests for all Hosts that use the same Disk Pool. DataCore recommend therefore that all Disk Pools have 'dedicated' physical disks used just for storing the primary and secondary Disk Pool Catalogs and that these physical disks are as fast as possible. As the Catalog is located within the first 1GB of the physical disk used to store it and as there is a minimum Disk Pool requirement of any physical disk to have enough free space to allocate at least one SAU, that this 'dedicated' physical disk be 2GB in size; 1GB for the Catalog itself and 1GB for the largest SAU possible within the disk pool (see previous section on Storage Allocation Unit size in this chapter). Page 31

32 Disk Pools How the Catalog location is managed during Disk Decommissioning If the physical disk that holds the primary copy of the Catalog is removed then the backup copy on the remaining second physical disk in the Disk Pool will be automatically 'promoted' to be the primary Catalog and, if available, a backup copy will be written to the 'third' physical disk added to the Disk Pool. If the physical disk that holds the backup copy of the Disk Pool is removed then a new backup copy of the Catalog will be written to the 'next available' physical disk in the Disk Pool. The location of the primary copy remains unchanged. It is not currently possible for a user to move the Catalog to a physical disk of their choice in a Disk Pool. Also see: How the Catalog location is managed during physical disk I/O failures If there is an I/O error when trying to update or read from the primary Catalog then the backup Catalog will become the new primary catalog and if there is another physical disk available then that will now become the new backup catalog location. Storage Allocation Unit size The SAU size is chosen at the time the Disk Pool is created and cannot be changed - which is why it is not always easy to apply some of these best practices to existing Disk Pools. Each SAU represents a number of contiguous Logical Block Addresses (LBA) equal to its size and, once allocated, will be used for further reads and writes within the LBA range it represents to a Virtual Disk's storage source. Any time a write I/O is sent by the Host to a LBA not able to be allocated within existing SAU's for that Virtual Disk a new SAU is allocated by the Disk Pool. The amount of space taken in Disk Pool's Catalog (see previous section) for each allocated SAU is the same regardless of the size of the SAU that was chosen when the Disk Pool was created. As the Catalog has a theoretical maximum size, it means that the larger the SAU size chosen, the larger the amount of physical disk that can be added into a Disk Pool. As each SAU is allocated, the Disk Pool's Catalog is updated which. The smaller the SAU size the more likely it is that any new write I/O will be outside of the LBA range of already allocated SAUs and so the more likely that the Catalog will need to be updated. In the previous section - The Disk Pool Catalog it is recommended to have the Catalog on the fastest disk possible so as to be able to complete any Catalog updates as fast as possible, likewise it is recommended to have the largest SAU size as possible currently this is 1GB to make the chance of future writes being outside of the range of already allocated SAU LBAs less likely. There are however exceptions to this recommendation see section 'Disk Pools and Snapshot' in the Snapshot chapter on page 41 and also refer to DataCore's Host Configuration guide which may also offer particular advice for your Host's operating system and Disk Pools (e.g. Excessive SAU usage when formatting Linux filesystems). Page 32