The Host Server. Linux Configuration Guide. October 2017

Transcription

1 The Host Server Linux Configuration Guide October 2017 This guide provides configuration settings and considerations for SANsymphony Hosts running Linux. Basic Linux administration skills are assumed including how to connect to iscsi and/or Fibre Channel Storage Array target ports as well as the processes of discovering, mounting and formatting a disk device.

2 Table of contents Changes made to this document 3 Compatibility lists 4 Oracle VM Server 4 RedHat Enterprise Linux 5 SUSE Linux Enterprise Server 6 Ubuntu 7 Other Linux distributions 9 The DataCore Server's settings 10 The Linux Host's settings 13 Operating system settings 13 Multipath configuration settings 14 SAP HANA 18 Known issues 19 All Linux distributions 20 Ubuntu 20 Appendix A 21 Preferred Server & Preferred Path settings 21 Appendix B 23 Configuring Disk Pools 23 Appendix C 24 Reclaiming storage 24 Previous changes 27 Page 2

3 Changes made to this document The most recent version of this document is available from here: All changes since August 2017 Added Compatibility lists - Red Hat Enterprise Linux RedHat Enterprise Linux 7.4 This version is currently considered 'Not Qualified' for SANsymphony 10.x versions and 'Not Supported' for SANsymphony-V 9.0 PSP 4 Update 4 (and earlier). SUSE Linux Enterprise Server 12.0 SP 3 This version is currently considered 'Not Qualified' for SANsymphony 10.x versions and 'Not Supported' for SANsymphony-V 9.0 PSP 4 Update 4 (and earlier). Ubuntu 17.x This version is currently considered 'Not Qualified' for SANsymphony 10.x versions and 'Not Supported' for SANsymphony-V 9.0 PSP 4 Update 4 (and earlier). All previous changes Please see page 27 Page 3

4 Compatibility lists Oracle VM Server SANsymphony 9.0 PSP 4 Update 4 (1) 10.0 (all versions) Version With ALUA Without ALUA With ALUA Without ALUA 3.3 and earlier Not Supported Not Supported Not Supported Not Supported 3.4.x Not Supported Not Supported Not Qualified Qualified (2) Notes: Qualified vs. Not Qualified vs. Not Supported See page 9 for definitions. DataCore Server Front-End Port connections Fibre Channel is supported but only with SANsymphony versions 10.0 PSP 6 or greater, iscsi connections are considered 'Not qualified'. Multipath Tools Use Multipathing Tools version el6.x86_64 or later earlier versions are considered 'Not Qualified'. Oracle VM Manager Qualification by DataCore was done without using Oracle's VM Manager. SCSI UNMAP SCSI UNMAP is supported. Reclaiming storage from DataCore Disk Pools See Appendix C: 'Reclaiming Storage' on page 24 for version-specific 'how to' instructions. 1 SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications 2 SANsymphony 10.0 PSP 6 or greater with Fibre Channel connections only. ISCSI is considered not qualified. Page 4

5 RedHat Enterprise Linux SANsymphony 9.0 PSP 4 Update 4 (1) 10.0 (all versions) Version With ALUA Without ALUA With ALUA Without ALUA 5.10 or earlier Not Supported Not Supported Not Supported Not Supported Qualified Qualified Not Qualified Not Qualified Not Qualified Not Qualified Qualified Not Qualified Not Supported Not Supported Not Qualified Not Qualified 7.0 Not Supported Not Supported Qualified Not Qualified 7.1 Not Supported Not Supported Not Qualified Not Qualified Not Supported Not Supported Qualified (2) Not Qualified 7.4 Not Supported Not Supported Not Qualified Not Qualified Notes: Qualified vs. Not Qualified vs. Not Supported See the page 9 for definitions. DataCore Server Front-End Port connections Fibre Channel and iscsi are supported. Multipath Tools Always use the most current version of Multipathing Tools available for your version of RHEL. SCSI UNMAP SCSI UNMAP is supported. Reclaiming storage from DataCore Disk Pools See Appendix C: 'Reclaiming Storage' on page 24 for version-specific 'how to' instructions. 1 SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications 2 Qualified with SANsymphony version 10.0 PSP 6 or greater; earlier versions are still considered 'not qualified'. Page 5

6 Linux compatibility lists SUSE Linux Enterprise Server SANsymphony 9.0 PSP 4 Update 4 (1) 10.0 (all versions) Version With ALUA Without ALUA With ALUA Without ALUA 10.x or earlier Not Supported Not Supported Not Supported Not Supported 11.0 (no SP) Not Supported Qualified Not Supported Not Supported 11.0 SP 1 Not Supported Not Supported Not Supported Not Supported 11.0 SP 2 Qualified Qualified Not Qualified Not Qualified 11.0 SP 3 Not Qualified Not Qualified Qualified Not Qualified 11.0 SP 4 Not Supported Not Supported Qualified (2) Not Qualified 12.0 Not Supported Not Supported Not Qualified Not Qualified 12.0 SP 1 Not Supported Not Supported Qualified (2) Not Qualified 12.0 SP 2 3 Not Supported Not Supported Not Qualified Not Qualified Notes: Qualified vs. Not Qualified vs. Not Supported See the page 9 for definitions. DataCore Server Front-End Port connections Fibre Channel and iscsi are supported except for SLES 11.0 SP 4 and 12.0 SP 1 where iscsi connections are 'Not qualified'. Multipath Tools For SLES 12.x use Multipathing Tools version git1.656f8865 or later earlier versions are considered 'Not Qualified'. For all other SLES versions before 12.x use the most current version available for your distribution. SCSI UNMAP SCSI UNMAP is not supported. Reclaiming storage from DataCore Disk Pools See Appendix C: 'Reclaiming Storage' on page 24 for version-specific 'how to' instructions. 1 SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications 2 Fibre Channel only iscsi FE connections are still considered 'Not Qualified'. Page 6

7 Linux compatibility lists Ubuntu This following table specifically applies only to the Linux distribution provided by Canonical Ltd. SANsymphony 9.0 PSP 4 Update 4 (3) 10.0 (all versions) Version With ALUA Without ALUA With ALUA Without ALUA 13.x and earlier Not Supported Not Supported Not Supported Not Supported LTS Not Supported Not Qualified Not Supported Qualified Not Supported Not Qualified Not Supported Not Supported 15.x Not Supported Not Supported Not Supported Not Qualified 16.x Not Supported Not Supported Not Qualified Not Qualified 17.x Not Supported Not Supported Not Qualified Not Qualified Notes: Qualified vs. Not Qualified vs. Not Supported See the page 9 for definitions. DataCore Server Front-End Port connections Fibre Channel and iscsi connections are both supported. Multipath Tools Use Multipath Tools version git1.656f8865 or later - earlier versions are considered 'Not Qualified'. Reclaiming storage from DataCore Disk Pools See Appendix C: 'Reclaiming Storage' on page 24 for version-specific 'how to' instructions. 3 SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications Page 7

8 Linux compatibility lists Qualified vs. Not Qualified vs. Not Supported Qualified This combination has been tested by DataCore and all the host-specific settings listed in this document applied using non-mirrored, mirrored and Dual Virtual Disks. Not Qualified This combination has not yet been tested by DataCore using Mirrored or Dual Virtual Disks types. DataCore cannot guarantee 'high availability' (failover/failback, continued access etc.) even if the host-specific settings listed in this document are applied. Self-qualification may be possible please see Technical Support FAQ #1506 Mirrored or Dual Virtual Disks types are configured at the users own risk; however, any problems that are encountered while using Linux versions that are 'Not Qualified' will still get root-cause analysis. Non-mirrored Virtual Disks are always considered 'Qualified' - even for 'Not Qualified' combinations of Linux/SANsymphony. Not Supported This combination has either failed 'high availability' testing by DataCore using Mirrored or Dual Virtual Disks types; or the operating System's own requirements/limitations (e.g. age, specific hardware requirements) make it impractical to test. DataCore will not guarantee 'high availability' (failover/failback, continued access etc.) if the host-specific settings listed in this document are applied. Mirrored or Dual Virtual Disks types are configured at the users own risk. Self-qualification is not possible. Mirrored or Dual Virtual Disks types are configured at the users own risk; however, any problems that are encountered while using Linux versions that are 'Not Supported' will get best-effort Technical Support (e.g. to get access to Virtual Disks) but no root-cause analysis will be done. Non-mirrored Virtual Disks are always considered 'Qualified' even for 'Not Supported' combinations of Linux/SANsymphony. Linux versions that are End of Life For versions that are listed as 'Not Supported', self-qualification is not possible. For versions that are listed as 'Not Qualified', self-qualification may be possible if there is an agreed support contract with your Linux Vendor as well. Please contact DataCore Technical Support before attempting any self-qualification. For any problems that are encountered while using Linux versions that are EOL with DataCore Software, only besteffort Technical Support will be performed (e.g. to get access to Virtual Disks). Root-cause analysis will not be done. Non-mirrored Virtual Disks are always considered 'Qualified'. Page 8

9 Other Linux distributions Linux distributions that are not listed in this document are considered 'Not Qualified' for SANsymphony versions 10.x and 'Not Supported' for SANsymphony-V versions 9.0 PSP 4 Update 4 or earlier. Self-qualification may be possible please see Technical Support FAQ #1506 Non-mirrored Virtual Disks are always considered 'Qualified' - even for 'Not Qualified' or 'Not Supported' combinations of Linux/SANsymphony. Page 9

10 The DataCore Server's settings These are the Host-specific settings that need to be configured directly on the DataCore Server. Also see: Video: Configuring Linux Hosts to use DataCore Virtual Disks Operating system type See the Registering Hosts section from the SANsymphony Help: Oracle VM Server When registering the Host choose the 'Linux (all other distributions)' menu option. RedHat Enterprise Linux When registering the Host choose the 'Linux (all other distributions)' menu option. SUSE Linux Enterprise Server When registering the Host choose the 'Linux SUSE Enterprise Server 11 ' menu option. Ubuntu Linux When registering the Host choose the 'Linux (all other distributions)' menu option. Port roles Ports used for serving Virtual Disks to Hosts should only have the Front End (FE) role enabled. Mixing other Port Role types may cause unexpected results as Ports that only have the FE role enabled will be turned off when the DataCore Server software is stopped (even if the physical server remains running). This helps to guarantee that any Hosts do not still try to access FE Ports, for any reason, once the DataCore Software is stopped but where the DataCore Server remains running. Any Port with the Mirror and/or Back End role enabled do not shut off when the DataCore Server software is stopped but still remain active. Page 10

11 The DataCore Server's settings Multipathing support The Multipathing Support option should be enabled so that Mirrored Virtual Disks or Dual Virtual Disks can be served to Hosts from all available DataCore FE ports. Also see the Multipathing Support section from the SANsymphony Help: Webhelp/Hosts.htm Non-mirrored Virtual Disks and Multipathing Non-mirrored Virtual Disks can still be served to multiple Hosts and/or multiple Host Ports from one or more DataCore Server FE Ports if required; in this case the Host can use its own multipathing software to manage the multiple Host paths to the Single Virtual Disk as if it was a Mirrored or Dual Virtual Disk. Note: Hosts that have non-mirrored Virtual Disks served to them do not need Multipathing Support enabled unless they have other Mirrored or Dual Virtual Disks served as well. Asymmetrical Logical Unit Access (ALUA) support The ALUA support option should be enabled if required and if Multipathing Support has been also been enabled (see above). Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA. More information on 'Preferred Servers' and 'Preferred Paths' used by the ALUA function can be found on in Appendix A on page 21. Serving Virtual Disks to the Hosts for the first time DataCore recommends that before serving Virtual Disks for the first time to a Host, that all DataCore Front-End ports on all DataCore Servers are correctly discovered by the Host first. Then, from within the SANsymphony Console, the Virtual Disk is marked Online, up to date and that the storage sources have a host access status of Read/Write. Virtual Disks LUNs and serving to more than one Host or Port DataCore Virtual Disks always have their own unique Network Address Authority (NAA) identifier that a Host can use to manage the same Virtual Disk being served to multiple Ports on the same Host Server and the same Virtual Disk being served to multiple Hosts. See the SCSI Standard Inquiry Data section from the online Help for more information on this: While DataCore cannot guarantee that a disk device's NAA is used by a Host's operating system to identify a disk device served to it over different paths generally we have found that it is. And while there is sometimes a convention that all paths by the same disk device should always using the same LUN 'number' to guarantees consistency for device identification, this may not be technically true. Always refer to the Host Operating System vendor s own documentation for advice on this. Page 11

12 The DataCore Server's settings DataCore's Software does, however always try to create mappings between the Host's ports and the DataCore Server's Front-end (FE) ports for a Virtual Disk using the same LUN number (1) where it can. The software will first find the next available (lowest) LUN 'number' for the Host- DataCore FE mapping combination being applied and will then try to apply that same LUN number for all other mappings that are being attempted when the Virtual Disk is being served. If any Host-DataCore FE port combination being requested at that moment is already using that same LUN number (e.g. if a Host has other Virtual Disks served to it from previous) then the software will find the next available LUN number and apply that to those specific Host- DataCore FE mappings only. 1 The software will also try to match a LUN 'number' for all DataCore Server Mirror Port mappings of a Virtual Disk too, although the Host does not 'see' these mirror mappings and so this does not technically need to be the same as the Front End port mappings (or indeed as other Mirror Path mappings for the same Virtual Disk). Having Mirror mappings using different LUNs has no functional impact on the Host or DataCore Server at all. Page 12

13 The Linux Host's settings Operating system settings Fibre Channel device timeouts Do not specify any fibre channel-specific device (i.e. lpfc_devloss_tmo for Emulex or, qlport_down_retry for the QLogic) but use the values in the multipath configuration file (see the next page). Disk Timeouts Set the SCSI Disk timeout value to 80 seconds for any DataCore Virtual Disk devices. echo 80 > /sys/block/[disk_device]/device/timeout For example if the two DataCore Virtual Disks are using /dev/sda and /dev/sdb respectively: echo 80 > /sys/block/sda/device/timeout echo 80 > /sys/block/sdb/device/timeout This will change the SCSI Disk timeout value to 80 seconds for those particular disk devices. The SCSI Disk timeout can then be verified by running the cat command on the disk device directly cat /sys/block/sda/device/timeout Important Note on setting the SCSI Disk Device Timeout The method used here to set the SCSI Disk timeout may revert back to the system's default value after a reboot. For RHEL, Oracle and UBUNTU Linux users please contact your vendor directly or consult their documentation for details on creating a rules file in /etc/udev/ for DataCore Virtual Disks. For SLES users please see: Note that this link (as of publication of this document) refers to 'SANsymphony-V 9' but applies to all versions of SANsymphony 10.x as we ll. Page 13

14 The Linux Host's settings Multipath configuration settings Multipath Tools versions Always refer to the compatibility - on page 4 - lists for your distribution's minimum version of multipath tools that have been qualified. Polling Interval In the defaults section of the multipath.conf file, the polling_interval must be set to 60 defaults { } polling_interval 60 This is a DataCore-required value which helps prevent excessive Host attempts to check for a Virtual Disk Storage Source s Host Access value, after a failure, is set as Offline. Smaller interval settings will interfere with overall Host performance. Do not add this parameter to the 'device' section (discussed on the next page) by mistake; else the setting will not work as expected. Blacklist exceptions Usually all storage vendor devices are specified under a separate device sub-section within the blacklist_exceptions section. blacklist_exceptions { } vendor product "DataCore" "Virtual Disk" Page 14

15 The Linux Host's settings The 'device' section ALUA-enabled Hosts Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA device { vendor "DataCore" product "Virtual Disk" path_checker tur prio alua failback 10 no_path_retry fail dev_loss_tmo infinity fast_io_fail_tmo 5 rr_min_io_rq 100 # Alternative option See notes below # rr_min_io 100 path_grouping_policy group_by_prio # Alternative policy - See notes below # path_grouping_policy failover } # optional - See notes below # user_friendly_names yes Without ALUA enabled Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA device { } vendor "DataCore" product "Virtual Disk" path_checker tur failback 10 dev_loss_tmo infinity fast_io_fail_tmo 5 no_path_retry fail # optional - See notes below # user_friendly_names yes Page 15

16 Device section notes Note: All entries listed are required by SANsymphony and are case sensitive. The Linux Host's settings dev_loss_tmo infinity Also requires the fast_io_fail_tmo 5 setting (see next). The dev_loss_tmo setting controls the length of time (normally indicated in seconds) before a PATH to a Virtual Disk, that has since become unavailable to the Linux Host, is removed from the Linux operating system. For example; when a DataCore Server is stopped or a PATH to a DataCore Server s Virtual Disk is set to Host Access: Not Allowed. Once a PATH to a Virtual Disk has been removed by the Linux operating system it can then only usually be re-established by manual intervention (e.g. user-initiated rescan on the Linux Host). The infinity value prevents the PATH from being removed. If the fast_io_fail_tmo 5 setting is not present in the multipath.conf file, the infinity setting is ignored and the dev_loss_tmo value defaults to 600 (10 minutes). Note: Some older kernels of RedHat Enterprise Linux and SUSE Linux Enterprise Server 11 SP2 and earlier do not support the infinity value and it will be ignored (and may also post an error in syslog). In that case, a default value - usually 600 seconds - will be applied instead. Use the cat command to verify that any DataCore Virtual Disks detected by the Linux Host are using the infinity value correctly. A simple example: sleshost3:~ # cat /sys/class/fc_remote_ports/rport-*\:*-*/dev_loss_tmo Note: The value of indicates a dev using the infinity setting. fast_io_fail_tmo This is required by the dev_loss_tmo infinity setting (see previous). Do not use any other value other than 5, otherwise the dev_loss_tmo setting will use a larger, default value (usually 600 seconds). failback This adds an extra wait period (10 seconds) that helps to prevent unnecessary failback attempts to a Virtual Disk whose Storage Source s Host Access value is still set as Not Allowed. Page 16

17 The Linux Host's settings no_path_retry Required for any Linux Hosts configured for ALUA and want to use the Preferred Server setting set to ALL. See Appendix A on page 21 about information regarding the ALL setting. path_checker This is a DataCore-required value. No other value should be used. path_grouping_policy group_by_prio or path_grouping_policy group_by_failover One of two possible values is required group_by_prio or failover - to avoid any other, non- DataCore, setting in the multipath.conf (i.e. from other storage arrays attached to the Linux Host) from taking precedence. Note: The failover value is unqualified for RHEL 6.5 and greater. In either case, make sure the Preferred Server setting on the DataCore Server is either set to 'Auto Select' or an explicit Server Name. See Appendix A on page 21 about information regarding the 'Auto Select' setting. rr_min_io_rq This is a DataCore-required value for Linux Hosts that are running kernels newer than Older versions must use the option rr_min_io 100 instead (see next). rr_min_io This is a DataCore-required value for Linux Hosts that are running kernels older than Newer versions must use the option rr_min_io_rq 100 instead (see previous). user_friendly_names This is an optional setting and simply specifies that the operating system should use the /etc/multipath/bindings file to assign a persistent and unique alias to the multipath device, in the form of mpathn. If this value is set to 'no' (or omitted completely) the operating system will use a WWID as an alias for the multipath device instead. Page 17

18 SAP HANA SAP HANA has been certified with SANsymphony. Please see the document 'SANsymphony with SAP HANA - Sizing Guidelines' for more information: Page 18

19 Known issues The following is intended to make DataCore Software customers aware of any issues that may affect performance, access or generally give unexpected results under certain conditions when Linux Hosts are used with SANsymphony. Some of the issues here have been found during DataCore s own testing but many others are issues reported by DataCore Software customers, where a specific problem had been identified and then subsequently resolved. DataCore cannot be held responsible for incorrect information regarding Linux products. No assumption should be made that DataCore has direct communication with any of the Linux vendors regarding the issues listed here and we always recommend that users contact their own Linux vendor directly to see if there are any updates or fixes since they were reported to us. For Known issues for DataCore s own Software products, please refer to the relevant DataCore Software Component s release notes. Page 19

20 Known issues All Linux distributions Formatting a Virtual Disk may take longer than expected Since SANsymphony-V 9.0 PSP 4, Linux Hosts have been able to take advantage of SANsymphony s SCSI-UNMAP support - See the Appendix C - Reclaiming Storage' on page 24. However for RHEL and SLES Hosts, this will result in the mkfs command sending additional discard commands during the format process, resulting in longer format times. Ubuntu has not been tested with SCSI UNMAP so is at this time considered unqualified for SCSI UNMAP operations. Use the -K option while formatting to disable the discard command during formatting: and Refer to your own man page to be sure your installation supports this option. Ext3 filesystems will use excessive Disk Pool Storage Allocations The Ext3 filesystem will use significant amounts of Storage Allocation Units (SAU) during the Writing superblocks and filesystem accounting information phase of the filesystem's creation. Care must therefore be taken so as not to completely use all the SAUs from the Disk Pool; and if Ext3 is required, then use a small (i.e. 4MB) SAU size. Other filesystem types do not seem to exhibit this behavior and use only a few SAUs during filesystem creation. Also refer to Appendix B - Configuration Disk Pools on page 25. Ubuntu Manual rescans are needed to update previously failed paths for Ubuntu Hosts DataCore have not been able to get Ubuntu to automatically re-detect paths to mirrored Virtual Disks that have failed or have been removed (e.g. after stopping a DataCore Server) and are then subsequently made available again. Manual intervention is required. Use the 'multipath' command to establish which paths were previously failed (and are now available from the DataCore Server): multipath -ll Then use the 'echo' command to send an IOCTL to the disk device and force the operating system to update the path s status properly. For example echo 1 > /sys/block/sdc/device/rescan Alternatively download and install the 'scsitools' package and use the 'rescan-scsi-bus' command to re-establish the connection to the previously failed path. Page 20

21 Appendix A Preferred Server & Preferred Path settings See the Preferred Servers and Preferred Paths sections from the SANsymphony Help: Without ALUA enabled If Hosts are registered without ALUA support, the Preferred Server and Preferred Path settings will serve no function. All DataCore Servers and their respective Front End (FE) paths are considered equal. It is up to the Host s own Operating System or Failover Software to determine which DataCore Server is its preferred server. With ALUA enabled Setting the Preferred Server to Auto (or an explicit DataCore Server), determines the DataCore Server that is designated Active Optimized for Host IO. The other DataCore Server is designated Active Non-Optimized. If for any reason the Storage Source on the preferred DataCore Server becomes unavailable, and the Host Access for the Virtual Disk is set to Offline or Disabled, then the other DataCore Server will be designated the Active Optimized side. The Host will be notified by both DataCore Servers that there has been an ALUA state change, forcing the Host to re-check the ALUA state of both DataCore Servers and act accordingly. If the Storage Source on the preferred DataCore Server becomes unavailable but the Host Access for the Virtual Disk remains Read/Write, for example if only the Storage behind the DataCore Server is unavailable but the FE and MR paths are all connected or if the Host physically becomes disconnected from the preferred DataCore Server (e.g. Fibre Channel or iscsi Cable failure) then the ALUA state will not change for the remaining, Active Nonoptimized side. However, in this case, the DataCore Server will not prevent access to the Host nor will it change the way READ or WRITE IO is handled compared to the Active Optimized side, but the Host will still register this DataCore Server s Paths as Active Non-Optimized which may (or may not) affect how the Host behaves generally. Page 21

22 Appendix A - Preferred Server & Preferred Path settings In the case where the Preferred Server is set to All, then both DataCore Servers are designated Active Optimized for Host IO. All IO requests from a Host will use all Paths to all DataCore Servers equally, regardless of the distance that the IO has to travel to the DataCore Server. For this reason, the All setting is not normally recommended. If a Host has to send a WRITE IO to a remote DataCore Server (where the IO Path is significantly distant compared to the other local DataCore Server), then the WAIT times accrued by having to send the IO not only across the SAN to the remote DataCore Server, but for the remote DataCore Server to mirror back to the local DataCore Server and then for the mirror write to be acknowledged from the local DataCore Server to the remote DataCore Server and finally for the acknowledgement to be sent to the Host back across the SAN, can be significant. The benefits of being able to use all Paths to all DataCore Servers for all Virtual Disks are not always clear cut. Testing is advised. For Preferred Path settings it is stated in the SANsymphony Help: A preferred front-end path setting can also be set manually for a particular virtual disk. In this case, the manual setting for a virtual disk overrides the preferred path created by the preferred server setting for the host. So for example, if the Preferred Server is designated as DataCore Server A and the Preferred Paths are designated as DataCore Server B, then DataCore Server B will be the Active Optimized Side not DataCore Server A. In a two-node Server group there is usually nothing to be gained by making the Preferred Path setting different to the Preferred Server setting and it may also cause confusion when trying to diagnose path problems, or when redesigning your DataCore SAN with regard to Host IO Paths. For Server Groups that have three or more DataCore Servers, and where one (or more) of these DataCore Servers shares Mirror Paths between other DataCore Servers setting the Preferred Path makes more sense. So for example, DataCore Server A has two mirrored Virtual Disks, one with DataCore Server B, and one with DataCore Server C and DataCore Server B also has a mirrored Virtual Disk with DataCore Server C then using just the Preferred Server setting to designate the Active Optimized side for the Host s Virtual Disks becomes more complicated. In this case the Preferred Path setting can be used to override the Preferred Server setting for a much more granular level of control. Page 22

23 Appendix B Configuring Disk Pools See Creating Disk Pools and Adding Physical Disks from the SANsymphony Help: The smaller the SAU size, the larger the number of indexes are required, by the Disk Pool driver, to keep track of the equivalent amount of allocated storage compared to a Disk Pool with a larger SAU size; e.g. there are potentially four times as many indexes required in a Disk Pool using a 32MB SAU size compared to one using 128MB the default SAU size. As SAUs are allocated for the very first time, the Disk Pool needs to update these indexes and this may cause a slight delay for IO completion and might be noticeable on the Host. However this will depend on a number of factors such as the speed of the physical disks, the number of Hosts accessing the Disk Pool and their IO READ/WRITE patterns, the number of Virtual Disks in the Disk Pool and their corresponding Storage Profiles. Therefore, DataCore usually recommend using the default SAU size (128MB) as it is a good compromise between physical storage allocation and IO overhead during the initial SAU allocation index update. Should a smaller SAU size be preferred, the configuration should be tested to make sure that a potential increased number of initial SAU allocations does not impact the overall Host performance. Page 23

24 Appendix C Reclaiming storage Using SCSI UNMAP commands SANsymphony's support for SCSI UNMAP when used in conjunction the Linux fstrim command or the mount o discard option on certain file system types and allows Hosts to send 'all-zero' write I/O to a Virtual Disk and trigger SANsymphony's Automatic Reclamation feature. See below. Also see the following online resources: The Linux man-pages project RedHat s Storage Administration Guide: Section: 2.5. Discard unused blocks Always refer to your Linux Vendor to determine which file systems are supported for either the fstrim command and/or the mount o discard option for the version of RHEL or SLES. Note that using the mount o discard option may affect Host performance again refer to your Linux Vendor for their recommendations. SANsymphony's Automatic Reclamation feature DataCore Servers keep track of any 'all-zero' write I/O requests sent to Storage Allocation Units (SAU) in all Disk Pools. When enough 'all-zero' writes have been detected to have been passed down to an entire SAUs logical address space, that SAU will be immediately assigned as 'free' (as if it had been manually reclaimed) and made available to the entire Disk Pool for future (re)use. No additional 'zeroing' of the Physical Disk or 'scanning' of the Disk Pool is required. Important technical notes on Automatic Reclamation The Disk Pool driver has a small amount of system memory that it uses keep a list of all address spaces in a Disk Pool that are sent 'all-zero' writes; all other (non-zero) write requests are ignored by the Automatic Reclamation feature and not included in the in-memory list. Any all-zero write addresses that are detected to be physically 'adjacent' to each other from a block address point of view the Disk Pool driver will 'merge' these requests together in the list so as to keep the size of it as small as possible. Also as entire 'all-zeroed' SAUs are re-assigned Page 24

25 Previous changes back to the Disk Pool, the record of all its address spaces is removed from the in-memory list making space available for future all-zero writes to other SAUs that are still allocated. However if write I/O pattern of the Hosts mean that the Disk Pool receives all-zero writes to many, non-adjacent block addresses the list will require more space to keep track of them compared to all-adjacent block addresses. In extreme cases, where the in-memory list can no longer hold any more new all-zero writes (because all the allocated system memory for the Automatic Reclamation feature has been used) the Disk Pool driver will discard the oldest records of the all-zero writes to accommodate newer records of all-zero write I/O. Likewise if a DataCore Server is rebooted for any reason, then the in-memory list is completely lost and any knowledge of SAUs that were already partially detected as having been written with all-zeroes will now no longer be remembered. In both of these cases this can mean that, over time, even though technically an SAU may have been completely overwritten with all-zero writes, the Disk Pool driver does not have a record that cover the entire address space of that SAU in its in-memory list and so the SAU will not be made available to the Disk Pool but remain allocated to the Virtual Disk until any future all-zero writes happen to re-write the same address spaces that were forgotten about previously by the Disk Pool driver. In these scenarios, a Manual Reclamation will force the Disk Pool to re-read all SAUs and perhaps detect those now missing all-zero address spaces. See the section 'Manual Reclamation' on the next page for more information. Reclaiming storage without using fstrim For Linux Hosts that either do not support the fstrim command or do not have the mount o discard option set, a suggestion would be to create a sparse file of an appropriate size (If there is enough free space available in the file system) and then zero-fill it using the dd command. This example creates an empty file (called my_file ) of 2GB in size: fallocate --length my_file Then use the dd command to fill all of the unused file system space with 'all-zero' write I/O. dd if=/dev/zero of=my_file bs=1024 count= This I/O will then be detected by SANsymphony's Automatic Reclamation function (see previous page for more details). Page 25

26 Previous changes Also see: Performing Reclamation section from the SANsymphony Help: SANsymphony's Manual Reclamation feature Manual reclamation forces the Disk Pool driver to 'read' all SAUs currently assigned to a Virtual Disk looking for SAUs that contain only all-zero IO data. Once detected, that SAU will be immediately assigned as 'free' and made available to the entire Disk Pool for future (re)use. No additional 'zeroing' of the Physical Disk is required. Note that manual reclamation will create additional 'read' I/O on the Storage Array used by the Disk Pool, as this process runs at 'low priority' it should not interfere with normal I/O operations. However, caution is advised, especially when scripting the manual reclamation process. Manual Reclamation may still be required even when Automatic Reclamation has taken place (see the 'Automatic Reclamation' section on the previous page for more information). How much storage will be reclaimed? It is impossible to predict exactly how many Storage Allocation Units (SAUs) will be reclaimed. For reclamation of an SAU to take place, it must contain only all-zero block data over the entire SAU else it will remain allocated and this is entirely dependent on how and where the Host has written its data on the DataCore LUN. For example, if the Host has written the data in such a way that every allocated SAU contains a small amount of non-zero block data then no (or very few) SAUs can be reclaimed, even if the total amount of data is much less than the total amount of assigned SAUs. It may be possible to use the Host operating system s own defragmentation tools to move andy data that is spread out over the DataCore LUN so that it ends up as one or more large areas of contiguous non-zero block addresses. This might then leave the the DataCore LUN with SAUs that now only have all-zero data on them and that can then be reclaimed. However care should be taken that the act of defragmenting the data itself does not cause more SAU allocation as the block data is moved around (i.e. re-written to new areas on the DataCore LUN) during the re-organization. Page 26

27 Previous changes 2017 August Compatibility lists - Red Hat Enterprise Linux RHEL This version is 'Not Supported' for SANsymphony V 9.x and earlier and 'Not Qualified' for SANsymphony versions 10.x RHEL This version is now qualified for SANsymphony 10.0 PSP 6 and greater when using ALUA (earlier versions of SANsymphony are still considered 'not qualified'). May Compatibility lists - Red Hat Enterprise Linux RHEL This version is now qualified for SANsymphony 10.0 PSP 6 and greater when using ALUA April Compatibility lists All Linux distributions A note about Multipathing Support has been added with regard to expected versions of 'Multipathing Tools' that should be installed. Compatibility lists - Red Hat Enterprise Linux RHEL 7.3 This version is 'not supported' for SANsymphony-V 9.x and 'not qualified' for SANsymphony 10.x. Compatibility lists - SUSE Linux Enterprise Server SUSE Linux Enterprise Server 12.0 SP 2 This version is 'not supported' for SANsymphony-V 9.x and 'not qualified' for SANsymphony 10.x. Compatibility lists - Ubuntu Ubuntu 15.x This version is 'not supported' for SANsymphony-V 9.0 PSP 4 Update 4 'Without ALUA'. Previously it was, incorrectly, marked as 'not qualified'. Removed SAP HANA certified configuration settings. The information has been moved to the 'SANsymphony with SAP HANA - Sizing Guidelines' document: February The Linux Host's settings Operating system settings Disk Timeouts Expanded the note under this section that refers to a URL from SUSE's own Knowledgebase that explains how to make the required SCSI Disk Device timeout setting permanent over a reboot. January Added Page 27

28 Previous changes Linux compatibility list - Oracle VM Server Version 3.4 has now been qualified with SANsymphony-V 10.0 PSP 6 or greater December Added The DataCore Server's settings Added link: Video: Configuring Linux Hosts to use DataCore Virtual Disks November Appendix C - Reclaiming storage Automatic and Manual reclamation These two sections have been re-written with more detailed explanations and technical notes. October Linux compatibility lists SUSE Linux Enterprise Server 11.0 SP 4 SUSE Linux Enterprise Server 12.0 SP 1 Both of these versions are now 'Supported' using ALUA with Fibre Channel Front-end connections. Note: iscsi Front End connections are still considered 'Not Qualified' July This document has been reviewed for SANsymphony 10.0 PSP 5. No other updates were required. April Added Linux compatibility list Red Hat Enterprise Linux Versions Red Hat Enterprise Linux Versions SUSE Linux Enterprise Server 12 (no Service Pack) and 12 Service Pack 1 January Linux compatibility list Red Hat Enterprise Linux Version Previously this version was listed as 'not qualified' with ALUA enabled Hosts and SANsymphony-V 9.x. This has now been changed to 'Not Supported with Mirrored or Dual Virtual Disks'. Red Hat Enterprise Linux Version 6.6 Previously this version was listed as 'Not Qualified' with ALUA enabled Hosts and SANsymphony-V 10.x. This version is now 'Qualified' with ALUA enabled Hosts and SANsymphony-V 10.x. November Added Linux Applications SAP HANA 2015 Page 28

29 Previous changes This includes an example of a 3-node SAP HANA configuration. SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications September Added Linux Applications SAP HANA A new section has been added with settings specific to SAP HANA. Linux compatibility list SUSE Linux Enterprise Server 11.0 Service Pack 3 using ALUA are certified with SAP HANA versions SPS09 and SPS10. Multipath Configuration Settings The multipath.conf settings for all qualified Linux distributions are now identical so there is only one section for all Linux distributions listed here. The setting dev_loss_tmo infinity requires the additional setting fast_io_fail_tmo to be present in the multipath.conf file this was omitted previously from the SUSE Linux Enterprise Server requirements; and that the fast_io_fail_tmo value is set to 5 this was previously set to 30 for Red Hat Enterprise Linux requirements. June Added Linux compatibility list Ubuntu LTS has now been qualified with SANsymphony-V 10.x. Known Issues Ubuntu requires manual intervention to redetect failed paths to a DataCore Server. April Added Known Issues All RHEL or SUSE Linux specific known issues with SANsymphony-V will now be documented here. March Linux compatibility lists SUSE Linux Enterprise Server 11.0 with Service Pack 3 is now qualified with SANsymphony-V 10.x using ALUA-only settings. February Linux compatibility lists Red Hat Enterprise Linux 7.0 is now qualified with SANsymphony-V 10.x using ALUA-only settings. Multipath Configuration Settings Red Hat Enterprise Linux Added new entry in the multipath.conf file no_path_retry fail Page 29

30 Previous changes This is required for any Oracle RAC/GFS configured Hosts Hosts using ALUA with the Preferred Server setting configured for ALL November Linux compatibility lists the table for all Red Hat Enterprise Linux and SUSE Linux Enterprise Server Versions Single Virtual Disks are now always considered supported. July Linux compatibility lists the table for all Red Hat Enterprise Linux Versions It was previously listing versions only. Host Settings Disk Timeouts Added an example for use of the cat command to determine the current SCSI Disk Timeout and a note that some versions of Linux may revert to the default settings after a reboot and how to resolve this. June Linux compatibility lists the table for SANsymphony-V 10.x May Added Linux compatibility lists Red Hat Enterprise Linux 6.5 with ALUA is now qualified for use with SANsymphony-V 9.x Appendix A - Reclaiming Storage from Linux Hosts Added information for 'ATA Trim' commands and Automatic Reclamation from Disk Pools. Multipath Configuration Settings Red Hat Enterprise Linux only Added a new Multipath.conf entry requirement - fast_io_fail_tmo - which is required to support the dev_loss_tmo value of infinity else the value may default back to 10 minutes instead. NB: Check with your Vendor to make sure your specific Linux Kernel version supports these options. Red Hat Enterprise Linux and SUSE Enterprise Linux DataCore recommends that the Multipath.conf entry path_grouping_policy be group_by_prio instead of failover. NB: The failover option is considered unqualified for RHEL version 6.5. Added an optional Multipath.conf entry user_friendly_names which will create simpler, easier to read, disk device names for users to work with. NB: Check with your Vendor to make sure your specific Linux Kernel version supports this option. April Page 30

31 Previous changes This document combines all of DataCore s Linux-related information from older Technical Bulletins into a single document including: Technical Bulletin 2a: " Other Linux Hosts" Technical Bulletin 2b: "Redhat Enterprise Linux 6.x Hosts" Technical Bulletin 2c: "SUSE Linux Enterprise Server 11.x Hosts" Technical Bulletin 8: "Formatting Host s File Systems on Virtual Disks created from Disk Pools" Technical Bulletin 11: "Disk Timeout Settings on Hosts" Technical Bulletin 16: "Reclaiming Space in Disk Pools" Added Linux compatibility lists Added new tables to show which versions are explicitly qualified, unqualified and not supported with either SANsymphony-V 8.x or 9.x, and if the configuration is with or without ALUA enabled Hosts. Note that the minimum requirement for SANsymphony-V 8.x is now 8.1 PSP1 Update 4 to guarantee expected behavior for qualified versions of Linux. Appendix A This section gives more detail on the Preferred Server and Preferred Path settings with regard to how it may affect a Host. Appendix B This section incorporates information regarding "Reclaiming Space in Disk Pools" (from Technical Bulletin 16) that is specific to Linux Hosts. Host Settings - SUSE Linux Enterprise Server with/without ALUA enabled dev_loss_tmo - For SLES 11 SP2 or greater, please make sure that multipath-tools or greater is installed for the infinity setting to work properly. Improved explanations to most of the required Host Settings and DataCore Server Settings generally and earlier Technical Bulletin 2a: " Other Linux Hosts" July 2013 Removed all references to SANmelody as this is now End of Life as of December Completely updated what is considered qualified, not qualified and not supported in this document. January 2013 the section for the multipath.conf sections specifically for the polling_interval line. July 2012 Added SANsymphony-V 9.x. the Notes section for the multipath.conf sections specifically for the getuid_callout line. May 2012 DataCore Server and Host minimum requirements. Removed all references to End of Life SANsymphony and SANmelody versions that are no longer supported as of December the configuration settings and entries for /etc/multipath.conf (with additional values and explanations) for all DataCore products. Page 31

32 Previous changes Users should re-check their existing configurations and make any appropriate changes. Removed all (out of date) step-by-step instructions on how to manage and configure/format disk devices on the Host. March 2011 Added SANsymphony-V December 2009 Initial Publication Technical Bulletin 2b: "Redhat Enterprise Linux 6.x Hosts" July 2013 Removed all references to RHEL version 5.3 as this were never tested with ALUA-enabled hosts and so may cause confusion. Please use Technical Bulletin 2a for this earlier version. This Bulletin is now only for RHEL versions 6.x. Host Requirements. Added additional information in the Notes section of the multipath.conf section. June 2013 Improved blacklist_exceptions example for multipath.conf. April 2013 Removed all references to SANmelody as this is now End of Life as of December settings for multipath.conf including additional settings and explanations. March 2013 RHEL versions 6.0, 6.1, 6.2 and 6.3 explicitly stated. January 2013 the section for the multipath.conf sections specifically for the polling_interval line. This was previously stated to go in the devices. This was incorrect, and should be added to defaults section instead. Technical Bulletin 2c: "SUSE Linux Enterprise Server 11.x Hosts" July 2013 This Bulletin is now only for SLES versions 11.x. Host Requirements. Completely updated what is considered qualified, not qualified and not supported in this document. Added additional information in the Notes section of the multipath.conf section. June 2013 Improved blacklist_exceptions example for multipath.conf. April 2013 Removed all references to SANmelody as this is now End of Life as of December settings for multipath.conf including additional settings and explanations. February 2013 Added notes for SLES 11 SP2; including an additional multipath.conf setting. that SLES 11 SP1 is no longer supported as this was found not work correctly in all failure conditions. Users on SLES 11 SP1 should update to SP2. January 2013 the section for the multipath.conf sections specifically for the polling_interval line. This was previously stated to go in the devices. This was incorrect, and should be added to defaults section instead. Page 32