PS Series Deployment: Configuring and Deploying the Dell EqualLogic Host Integration Toolkit for Linux

Transcription

1 TECHNICAL REPORT PS Series Deployment: Configuring and Deploying the Dell EqualLogic Host Integration Toolkit for Linux Abstract This Technical Report describes how to deploy and configure Multipath I/O using the Dell EqualLogic HIT for Linux with Dell EqualLogic PS Series storage arrays. TR1065 V1.0

2

3 Copyright 2011 Dell Inc. All Rights Reserved. Dell EqualLogic is a trademark of Dell Inc. All trademarks and registered trademarks mentioned herein are the property of their respective owners. Possession, use, or copying of the documentation or the software described in this publication is authorized only under the license agreement specified herein. Dell, Inc. will not be held liable for technical or editorial errors or omissions contained herein. The information in this document is subject to change. March

4 Current Customers Please Note: You may not be running the latest versions of the tools and software listed below. If you are under valid warranty or support agreements for your PS Series array, you are entitled to obtain the latest updates and new releases as they become available. To learn more about any of these products, contact your local sales representative or visit the Dell EqualLogic site at To set up a Dell EqualLogic support account to download the latest available PS Series firmware and software kits visit: iii

5 PREFACE Thank you for your interest in Dell EqualLogic PS Series storage products. We hope you will find them intuitive and simple to configure and manage. PS Series arrays optimize resources by automating volume and network load balancing. Additionally, PS Series arrays offer all-inclusive array management software, host software, and free firmware updates. The features and products described next are available at no additional cost. PS Series Software Firmware: Installed on each array, PS Series firmware software allows you to manage your storage environment and provides capabilities such as volume snapshots, cloning, and replication to ensure data hosted on the arrays is protected in the event of an error or disaster. Group Manager GUI: Provides a graphical user interface for managing a group. Group Manager CLI: Provides a command line interface for managing a group. Manual Transfer Utility (MTU): Runs on Windows and Linux systems and enables you to use physical media to securely transfer large amounts of data to a replication partner, facilitating replication and preventing network congestion. SAN HeadQuarters (SANHQ): Provides centralized monitoring, historical performance trending, and event reporting for multiple PS Series groups. Host Software for Linux Remote Setup Wizard Command Line Interface (RSWCLI): Discovers and initializes PS Series arrays, configures and manages access to PS Series arrays, and configures and manages multipathing. Multipath device configuration components: Provides the EqualLogic Connection Manager (ehcmd) daemon to manage multipath devices, a Device Mapper kernel module (dmswitch) to optimize routing of multipathing devices, and a command line interface (ehcmcli) that allows users to review the diagnostic state of EqualLogic multipathing. EqualLogic Host Performance and Tuning Tool Suite (eqltune), a utility used to validate configurable parameters against practices recommended by Dell. Host Software for Windows Remote Setup Wizard (RSW): Initializes new PS Series arrays, configures host connections to PS Series groups, and configures and manages multipathing. Remote Setup Wizard Command Line Interface (RSWCLI): Provides a command-line alternative to the Remote Setup Wizard. Multipath I/O Device Specific Module (MPIO DSM): Includes a connection awareness module that understands PS Series network load balancing and facilitates host connections to PS Series volumes. VSS and VDS Provider Services: Allows 3rd party backup software vendors to perform offhost backups. Auto-Snapshot Manager/Microsoft Edition (ASM/ME): VSS Requestor interface that uses PS Series snapshots, cloning, and replication to provide point-in-time application-consistent iv

6 protection of critical data for supported applications (including SQL Server, Exchange Server, Hyper-V, and NTFS file shares). PowerShell Tools: Provides a comprehensive set of PowerShell cmdlets for managing one or many PS Series groups. Host Software for VMware Host Integration Tools for VMware - Auto-Snapshot Manager/VMware Edition (ASM/VE): Integrates with VMware Virtual Center and PS Series snapshots to allow administrators to enable Smart Copy protection of Virtual Center folders, datastores, and virtual machines. - EqualLogic Datastore Manager: Allows administrators to create and manage datastore on EqualLogic PS Series arrays from within vcenter. - Virtual Desktop Deployment Utility: Automates the deployment of virtual desktops in a VMware View environment. Storage Adapter for Site Recovery Manager (SRM): Allows SRM to understand and recognize PS Series replication for full SRM integration. Multipathing Extension Module (MEM): Provides enhancements to existing VMware multipathing functionality. Includes a connection awareness module that understands PS Series network load balancing and facilitates host connections to PS Series volumes. v

7 Table of Contents Revision Information... vii Introduction... 1 HIT for Linux: Features and Benefits... 1 Dell EqualLogic Multipath I/O Overview... 1 Dell EqualLogic MPIO Software... 3 Getting Started... 5 Installing the Software... 7 Resolving Critical Issues... 9 Verifying the Software Installation Managing the Configuration Using HIT for Linux Discover and Login to a PS Series Target EqualLogic MPIO Session Activity Log off a Volume Best Practices Network settings: Jumbo Frames and Flow Control iscsi Logins Load Balancing Policy Maximum Number of Sessions per Volume Configuration File Summary Technical Support and Customer Service vi

8 REVISION INFORMATION The following table describes the release history of this Technical Report. Report Date Document Revision 1.0 March 2011 Initial Release The following table shows the software and firmware used for the preparation of this Technical Report. Vendor Model Software Revision Red Hat Red Hat Enterprise Linux TM RHEL 5 Update 5 Dell TM EqualLogic TM PS Series Firmware V5.0.2 The following table lists the documents referred to in this Technical Report. All PS Series Technical Reports are available on the Customer Support site at: Vendor Document Title Dell EqualLogic Host Integration Tools for Linux Release Notes Dell EqualLogic Host Integration Tools for Linux Installation and User Guide Dell EqualLogic Deploying Red Hat Enterprise Linux with Dell EqualLogic PS Series Arrays Red Hat Red Hat Red Hat Enterprise Linux 5 DM Multipath Configuration and Administration US/Red_Hat_Enterprise_Linux/5/pdf/DM_Multipath/Red_Hat_Enterprise_Linux- 5-DM_Multipath-en-US.pdf Red Hat Enterprise Linux 6 DM Multipath Configuration and Administration US/Red_Hat_Enterprise_Linux/6/pdf/DM_Multipath/Red_Hat_Enterprise_Linux- 6-DM_Multipath-en-US.pdf vii

9

10 INTRODUCTION This technical report documents the features and usage of the Dell EqualLogic Host Integration Toolkit for Linux (HIT for Linux). It introduces HIT for Linux feature set, describes iscsi multipathing concepts, and then guides you through installation, configuration and usage of HIT for Linux. The final section of the report discusses Dell s recommended best practices for Linux iscsi server configuration. Note that this technical report assumes you are familiar with iscsi concepts and Linux server administration. For more introductory information, see the technical report titled Deploying Red Hat Enterprise Linux with Dell EqualLogic PS Series Arrays. Please refer to the Revision Information section of this document for the URL. HIT for Linux: Features and Benefits HIT for Linux is designed to provide these features and corresponding benefits: Reduced configuration and administration effort: HIT for Linux automates multipath configuration, eliminating the need to manually configure several system aspects. For example, there is no need to manually manage iscsi interface records with the iscsiadm command. In addition, the need to manually add device information to the multipath.conf file is eliminated. Also, HIT for Linux automatically implements a persistent naming scheme reducing the administration effort to add new iscsi volumes to a server. By reducing configuration and administration efforts, existing servers can be reconfigured more quickly and new servers can be deployed in less time. Intelligent routing and automated multipath I/O session management: HIT for Linux automatically establishes and manages iscsi sessions to each volume and intelligently directs I/O traffic. In multi-array member storage pools the PS Series automatically distributes the volume across multiple arrays. HIT for Linux maintains volume layout information which is used to route I/O requests intelligently. The software dynamically adapts to the number of available routes on a per volume basis, providing failover capabilities and improved performance. By automatically managing iscsi sessions, maintaining the routing table and leveraging page location knowledge HIT for Linux will dynamically direct I/O traffic to the optimal route. Automated analysis of the Linux servers iscsi settings: HIT for Linux analyzes a Linux system and produces a report which guides the user to implement an optimal iscsi configuration. The tool analyzes and reports on numerous system settings, comparing them to Dell s recommended practices. This feature greatly simplifies Linux iscsi tuning for administrators. Dell EqualLogic Multipath I/O Overview Multipath I/O (MPIO) framework uses redundant physical connections to shared storage. Having multiple host connections, switches and SAN interfaces all help to eliminate single points of failure. Using MPIO, servers can send multiple I/O streams to SAN volumes concurrently. MPIO also provides increased redundancy and can improve performance of application data hosted on the SAN. See [Figure1] and [Table1]. 1

11 Figure 1: MPIO Configuration and Components Table 1: EqualLogic MPIO Component Description 1 The dm-switch kernel module performs I/O path selection and error handling 2 The EqualLogic host connection manager daemon (ehcmd) manages iscsi sessions 3 Two or more 1 Gb or 10 Gb Ethernet NICs 4 Two or more redundant physical paths to redundant network switches 5 Two or more redundant physical paths provide a network switch interlink (ISL) 6 Two or more redundant physical paths from the network switches are connected to the separate redundant controllers in each Member in a PS Series Group. 7 Each Member in the Group has at least two physical paths to the network switches that are connected to separate redundant controllers. 2

12 Dell EqualLogic MPIO Software The Dell EqualLogic Multipath I/O (MPIO) software is designed to deliver: Automatic connection management Automatic path failure detection and path failover Automatic load balancing across paths Support for multiple connections to a single iscsi target (volume) Increased bandwidth Reduced network latency Easy installation and management The Dell EqualLogic HIT for Linux MPIO software consists of two components: 1. The EqualLogic Host Connection Manager daemon, ehcmd, which monitors the iscsi session state and the configuration of the Linux server and PS Series group. Running in the background, the ehcmd daemon uses the Open-iSCSI management tool (iscsiadm) to add, remove, or modify iscsi sessions to maintain an optimal number of iscsi sessions. It also gathers information on the volume layout from the PS Series group. 2. A loadable kernel module, dm-switch, which implements a new Device Mapper target to the multipath devices. Based on the volume layout on the array members, the dmswitch module routes each I/O to the optimal path within the PS Series group. Dell EqualLogic MPIO is easy to install as part of the Host Integration Tool Kit for Linux. The Host Integration Tool Kit is included with the purchase of a PS Series array or downloadable from at no extra cost. 3

13 Figure 2 shows how an I/O operation on the Linux host using the HIT for Linux MPIO software communicates with the PS Series Group. The figure shows the internal implementation in the Linux storage device stack. The following paragraph describes the components in Figure 2, by callout number (1 7). Note that the identifying numbers are component numbers, not step numbers. The numbers do not necessarily represent the order in which the operations are performed. The Linux host software, either through a filesystem operation or direct application I/O, accesses the dm-switch device (1). HIT for Linux uses the Device Mapper to build a two-tier device structure for each volume (2). The ehcmd daemon builds a single top-tier device for each iscsi volume (3). Multiple dm-multipath devices are built for each volume, (at least) one for each group member that holds the volume (4). Each iscsi session has a corresponding SD block device (5). The ehcmd daemon dynamically provides automatic iscsi session creation and management (6). The volume data resides on the PS Series arrays (7). Figure 2: Multipath Device Configuration Logical View 4

14 Figure 3 illustrates how multiple I/O streams are concurrently routed through the host s iscsi sessions directly to the member (array) within the PS Series Group which will be servicing them. Using the Dell EqualLogic MPIO software, the best path for each I/O is selected based on knowledge about the volume layout on the PS Series Group. The Dell EqualLogic MPIO software uses iscsiadm to automatically manage these iscsi sessions and then routes I/O directly to the member (array) on which it will be serviced. Figure 3: Dell EqualLogic MPIO Managing iscsi Sessions and I/O Routing GETTING STARTED Before you get started with installing HIT for Linux, it is a good idea to plan your networking configuration and review some aspects of your Linux server environment. For optimal SAN performance and redundancy, Dell recommends that you dedicate more than one network interface on your Linux servers to iscsi SAN traffic. Additionally, SAN traffic should be on a dedicated network, separate from your LAN network. Before beginning the software installation, you should verify the following conditions are met: Linux server to PS Series SAN connectivity. Verify that each Ethernet interface being used for SAN traffic can access the PS Series group IP address. You can do so by using the ping command, directing it to use a specific network interface with the I option: 5

15 Software pre-requisites - These are listed in [Table 2] and also discussed in the Host Integration Tools for Linux Installation and User Guide. Table 2 Package RHEL 5.5 Min.Version RHEL 6.0 Min.Version Description OpenSSL 0.9.8e Cryptographic APIs encrypt passwords stored by rswcli libpcap Used by the Remote Setup Wizard array initialization protocol Open-iSCSI initiator iscsi-initiatorutils devicemappermultipath User mode tools for Device Mapper multipath configuration and management DKMS Dynamic Kernel Module Support; used to build and install the dm-switch kernel module Python Executes configuration scripts, including eqltune Be sure that the pre-requisites are met before proceeding with the HIT for Linux installation. Failing to do so will cause the installation process to abort. Table 2 provides a list of the prerequisite package names and version numbers, many of which may already be installed on your Linux system. You can verify the installation status and version number of packages using a command sequence similar to this: Note the Dynamic Kernel Module Support (dkms) package requirement. While the HIT for Linux software comes packaged by RPM, the EqualLogic iscsi Multipathing software needs to be built against the running kernel. The DKMS package is used during the HIT for Linux installation procedure to compile the EqualLogic supplied kernel module and install it. DKMS is not part of the Red Hat base repository. DKMS can be downloaded from Download and install the latest stable RPM package. Multipathd configuration - The EqualLogic iscsi Multipathing software does not require the Linux Multipath daemon (multipathd). Do not run multipathd before configuring HIT for Linux MPIO devices. If your environment requires the use of multipathd, be sure to blacklist EqualLogic devices in the multipath.conf file. For more information on how to blacklist 6

16 multipath devices, refer to the Red Hat Enterprise Linux DM Multipath Configuration and Administration document listed in the Revision Information section of this technical report. Installing the Software This section of the technical report describes the installation procedure for HIT for Linux. For more complete information please refer to the following product documentation: EqualLogic Host Integration Tools for Linux Release Notes EqualLogic Host Integration Tools for Linux Installation and Users Guide Please refer to the Revision Information section of this technical report for information on how to access these documents. HIT for Linux is packaged in RPM format, for both 32-bit and 64-bit systems, and supports the following Linux distributions: Red Hat Enterprise Linux 5.5 Red Hat Enterprise Linux 6 The HIT for Linux package, public key and documentation can be downloaded from the Dell EqualLogic support site ( Download HIT for Linux that matches your base operating system install, in both architecture (32 or 64 bit) and RHEL release version. Attempting to install the wrong package will cause the installation to fail, possibly with misleading messages (e.g. missing dependencies). Once the proper HIT for Linux package is downloaded and you have confirmed that the prerequisites are met, install the HIT for Linux package using the command line options appropriate for your package manager. This screenshot shows an excerpt of the install using yum. The public key (RPM-GPG-KEY-DELLEQL) and HIT for Linux package were previously downloaded from the EqualLogic support site: 7

17 The installation procedure will copy the EqualLogic tools and documentation to your system, install the dm-switch kernel module, start the connection manager daemon and check your systems iscsi configuration. The following EqualLogic tools will be installed in the /usr/sbin directory: rswcli remote setup wizard command line interface eqltune EqualLogic iscsi configuration tuning tool ehcmd EqualLogic host connection manager daemon ehcmcli EqualLogic host connection manager command line interface eqllog EqualLogic log collection tool In addition to man pages, the product documentation (Release Notes and Installation and Users Guide) will be installed in the /usr/share/doc/equallogic-host-tools-<version>/doc directory. The final operation performed by the installation script is to execute the EqualLogic iscsi configuration tuning tool (/usr/sbin/eqltune). This tool analyzes a Linux systems iscsi configuration for compliance with Dell s recommended practices, guiding the user to an optimal iscsi performance configuration. The utility produces a report which provides specific recommendations, along with instructions that guide users to create a more optimal iscsi server configuration. To provide concise output during the installation process, the eqltune tool is run in default mode. The default mode limits the eqltune output report to a summarized table of the findings. The output at the end of the installation procedure should look similar to the output below: Any settings flagged as Critical may prevent the proper operation of HIT for Linux software and need to be addressed immediately. To identify and resolve the issue, run eqltune using the 8

18 verbose flag (-v). This will produce many lines of output, so it is recommended to redirect the output to a file: # eqltune v > eqltune.report Resolving Critical Issues Several common Critical issues are listed in [Table 3] below, and are discussed in more detail in the HIT for Linux Release Notes. Table 3 SECTION AREA BEHAVIOR VALUE Sysctl Tunables NIC Settings ARP Flux arp_ignore=1 Sysctl Tunables NIC Settings Reverse Path Filtering arp_announce=2 Ethernet Devices ethx Generic Receive Offload off iscsi Settings iscsid.conf FastAbort No The report created by eqltune in verbose mode will not only describe the offending setting but also provide instructions on how to resolve the issue. Here is sample output from eqltune flagging the FastAbort setting as Critical and directing the user to edit the iscsid.conf file: 9

19 Here is another excerpt from the eqltune v output, this one reporting on the ARP response settings: Review the output from running eqltune v on your system. Once the Critical issues are resolved, you can re-run eqltune to confirm that the settings have been corrected. Then, proceed to the next step Verifying the Software Installation. If you find that you need more information about the analysis being performed by eqltune, there is online help in the form of an eqltune man page. The utility is designed to be self documenting. For additional information about the specific Linux system settings that are checked by the tool, review the output from the following command line: # eqltune vl Verifying the Software Installation After HIT for Linux install completes, you should verify that the software is properly installed. Key steps include checking the status of the dm-switch module and reviewing the EqualLogic host communication daemon diagnostic information. If the dm-switch module installed successfully you should see output similar to this: 10

20 The next screenshot shows two commands. The first verifies that the EqualLogic host connection manager daemon is running and the second communicates with the daemon, displaying diagnostic information: In this configuration the ehcmd daemon is running and managing three network interfaces (eth0, eth2, eth3). The connection manager has no active iscsi sessions and no volumes are being managed. If either of these verification commands produces significantly different output on your system (i.e. ehcmd is not running), the software installation failed. Please refer to the Troubleshooting section of Host Integration Tools for Linux Installation and Users Guide. A link to this document is provided in the Revision Information section of this technical report. Managing the Configuration Include and Exclude Subnets By default the EqualLogic host connection manager daemon will manage all available Ethernet interfaces. As a best practice, in many environments administrators will want to restrict the 11

21 connection manager to only manage the network adapters on the SAN subnet. Failing to do so will likely result in iscsi connection problems, such as timeouts during volume login operations. In our sample configuration, there are two subnets and three active Ethernet interfaces: eth0; eth2 and eth3. The X subnet (eth0) is dedicated to LAN traffic while the X subnet (eth2 and eth3) is dedicated to SAN traffic. Therefore, the x LAN subnet should be excluded from HIT for Linux MPIO software. This screenshot shows two commands. The first rswcli E performs the subnet exclusion while the second rswcli L lists the subnets included and excluded: 12

22 The output from the ehcmd d command confirms that the EqualLogic host connection manager is now managing two network interfaces (eth2 and eth3), rather than three: If there is a need to exclude a specific interface by IP address, the following syntax can be used in an environment using static IP addresses: # rswcli E network mask This command will exclude just this one IP address ( ). It is recommended that you use the rswcli P and ehcmcli d commands to confirm that the correct settings have been applied. Linux iscsiadm Records The EqualLogic MPIO software uses the Linux iscsiadm utility to automatically manage iscsi connections and interfaces (iface). These three commands demonstrate use of the iscsiadm utility to view portions of the open-iscsi database, displaying the node, session and iface records: [root@rh55 Desktop]# iscsiadm -m node iscsiadm: no records found! [root@rh55 Desktop]# iscsiadm -m session iscsiadm: No active sessions. [root@rh55 Desktop]# iscsiadm -m iface default tcp,<empty>,<empty>,<empty>,<empty> iser iser,<empty>,<empty>,<empty>,<empty> 13

23 eql.eth0_0 tcp,00:21:9b:8a:e2:97,<empty>,<empty>,<empty> eql.eth2_1 tcp,00:10:18:4a:66:fc,<empty>,<empty>,<empty> eql.eth3_1 tcp,00:10:18:4a:66:fe,<empty>,<empty>,<empty> eql.eth2_0 tcp,00:10:18:4a:66:fc,<empty>,<empty>,<empty> eql.eth0_1 tcp,00:21:9b:8a:e2:97,<empty>,<empty>,<empty> eql.eth3_0 tcp,00:10:18:4a:66:fe,<empty>,<empty>,<empty> The outputs from the first two commands indicate that there are no active iscsi node records or sessions. This is as expected since no volumes have been logged into. The third command displays the iscsi interface (iface) records. The iface records prefixed by eql. have been automatically created by the EqualLogic MPIO software. Two of these records (eql.eth0_0 and eql.eth0_1) are now obsolete, since we excluded the eth0 subnet. The EqualLogic MPIO software will automatically manage the iscsi ifaces, removing the obsolete interfaces after a few minutes. Administrators can also manually delete iface records with the iscsiadm utility, as seen below: These configuration changes leave the following ifaces: [root@rh55 Desktop]# iscsiadm -m iface default tcp,<empty>,<empty>,<empty>,<empty> iser iser,<empty>,<empty>,<empty>,<empty> eql.eth2_1 tcp,00:10:18:4a:66:fc,<empty>,<empty>,<empty> eql.eth3_1 tcp,00:10:18:4a:66:fe,<empty>,<empty>,<empty> eql.eth2_0 tcp,00:10:18:4a:66:fc,<empty>,<empty>,<empty> eql.eth3_0 tcp,00:10:18:4a:66:fe,<empty>,<empty>,<empty> Once the MPIO software is configured to use the proper subnets and the extraneous iface records are removed, you can proceed to the next section where we will discover and login to iscsi volumes. Using HIT for Linux In the previous sections of this technical report, HIT for Linux software was installed and configured. This section will walk you through connecting to a volume, monitoring MPIO sessions and disconnecting from a volume. In order to begin using HIT for Linux MPIO software, there needs to be at least one PS Series volume accessible from the Linux servers SAN Ethernet interfaces. Ensure that the volume(s) are configured with appropriate iscsi access specifying the servers iscsi initiator name, CHAP authentication or multiple IP addresses. Volume access control can be added to volumes through the PS Series Group Manager, either at the time of volume creation or afterwards. Discover and Login to a PS Series Target The iscsi volume discovery and login procedure is unchanged when using HIT for Linux MPIO software. The iscsiadm utility is still used to discover and login to volumes. It s worth noting that in its default mode, iscsiadm will login through all available iface records including those added by the EqualLogic MPIO software. This will result in unnecessary iscsi sessions being created, negatively impacted login performance and potentially straining SAN resources. 14

24 Directing iscsi login operations to use one iface is recommended. Since the EqualLogic MPIO software dynamically manages iscsi sessions, additional sessions will be automatically added and removed as required. In order to more efficiently perform iscsi login operations, the iscsiadm login commands should specify one iface record. Use the [-I iface_name] syntax in your login commands, as shown in the screenshot below. Once a volume is logged into, the EqualLogic MPIO software will create the required additional sessions based on your current configuration. An example of the recommended iscsiadm command syntax for volume discovery and login is shown here: [root@rh55 Desktop]# iscsiadm -m discovery -p t st :3260,1 iqn com.equallogic:0-8a0906-a884a4502- fdefc658d2e4d064-rh55fio :3260,1 iqn com.equallogic:0-8a0906-a884a4502- fdefc658d2e4d064-rh55fio :3260,1 iqn com.equallogic:0-8a0906-a884a4502- fdefc658d2e4d064-rh55fio :3260,1 iqn com.equallogic:0-8a0906-a884a4502- fdefc658d2e4d064-rh55fio1 The volume (rh55fio1) was discovered four times, once on each of the four iscsiadm iface records. To perform the login, use the following syntax: # iscsiadm m node T <iqn> -I iface_name -l This screenshot shows the iscsiadm login command using the eql.eth2_0 iface. The second iscsiadm command lists the session information. The volume (rh55fio1) is on a three array member storage pool and the EqualLogic MPIO software is managing six iscsi sessions: For additional information on techniques for managing iscsi logins, please refer to the Best Practices section this technical report. 15

25 EqualLogic MPIO Session Activity There are several techniques to monitor iscsi MPIO session activity including: # ehcmcli d (alternatively, use ehcmcli dv for more verbose information) In the PS Group Manager, View Connections (for the particular volume) # iscsiadm m session A screenshot of the host connection manager diagnostic output is shown here: The host connection manager diagnostic output is presented in three sections: Adapter List; Volume List and Summary. In our sample configuration there are two network interfaces (eth2 16

26 and eth3). We are connected to one volume (rh55fio1), which is in a storage pool with three PS Series array members. The connection manager is managing six iscsi sessions (3 arrays, 2 connections per array) to the volume. The IP addresses of the interfaces being used are listed, along with the corresponding Session numbers, device names and SCSI device numbers. The EqualLogic MPIO software created a dm-switch (top-tier) device and multipath devices for this volume. For an illustration of the two-tier device structure, refer to Figure 2 earlier in this document specifically items 3 and 4. The contents of the /dev/mapper directory are seen here: The dm-switch device name ends with the PS volume name (-rh55fio1). This is the persistent name that should be used by Linux server software applications. Users should only access the volume using the dm-switch device name. This is the same name as reported by ehcmcli d, under the Volume list, Device to mount: field. The other device names here (eql- *_a, eql-*_b, eql-*_c) represent the DM multipath devices. Those devices should never be accessed directly and are for use only by the EqualLogic MPIO software. 17

27 To create a new filesystem on the rh55fio1 volume, use the dm-switch device name. Then the new filesystem can be mounted on the server, as shown here: Log off a Volume The iscsi volume logoff procedure is unchanged when using HIT for Linux MPIO software. The Linux system shutdown procedure will handle iscsi volume logoff. In situations where you need to manually perform the logout, use one of the following approaches. The first technique restricts the logout to a specific iscsi volume, while the second attempts to logout of all iscsi volumes: # iscsiadm m node T <iqn> -u # iscsiadm m node U all As a best practice, always be sure that a filesystem is un-mounted before the associated iscsi volume sessions are logged out. In addition, never set an iscsi volume offline in the PS Group manager without first un-mounting the filesystem and disconnecting from the volume by logging out of the sessions with iscsiadm. 18

28 This screenshot shows the iscsiadm logoff command and corresponding output: This above output shows that each of the six iscsi sessions previously connected to this volume are now logged out. 19

29 The command outputs in the following screenshot confirm that the server is no longer connected to the volume: Best Practices This section of the technical report discusses several of Dell s recommended best practices for Linux iscsi server configuration. Some of these settings are analyzed and reported on by the EqualLogic tuning tool (eqltune). See the Host Integration Tools for Linux Installation and Users Guide for more information. Network settings: Jumbo Frames and Flow Control The eqltune utility analyzes a Linux systems iscsi and network settings and recommends settings for a more optimal iscsi configuration. In the area of Ethernet devices, eqltune checks for several settings including: Setting Category Value Generic Receive Offload (GRO) Critical off Flow Control Suggestion on 20

30 Jumbo Frames (MTU) Suggestion 9000 The GRO setting was discussed earlier in this technical report. It is reported as a critical issue and must be resolved before using HIT for Linux software. The flow control and jumbo frame settings are reported by eqltune in the category of Suggestion. If your network infrastructure supports either or both of these settings, then Dell s recommended best practice is to enable them on both the network switches and in the host operating system. The report produced by eqltune will flag these settings and provide instructions on how to enable them in Linux. This screenshot is an excerpt from an eqltune report showing the network settings for the eth2 device: iscsi Logins During periods of heavy iscsi login activity, Linux server and SAN resources can be stressed. This may happen during system reboots, if a large number of iscsi volumes are simultaneously being logged into. Configuration changes can be made to help alleviate the issue. There are two areas to consider: iscsiadm login command syntax Automatic login settings 21

31 As mentioned earlier in this document, it is recommended that iscsi logins be performed on one iface using the following syntax: # iscsiadm m node T <iqn> -I iface_name -l This technique limits the login operation to only be performed on one iscsi iface, rather than all of them. Depending on the number of volumes being logged into, along with the number of ifaces, this approach can significantly reduce the number of logins improving iscsi volume login responsiveness. Once the volume is logged into, the EqualLogic MPIO software will automatically create any additional iscsi sessions required for the configuration. The node.startup setting in the /etc/iscsi/iscsid.conf file determines if volumes will be automatically logged into during a system restart. This setting is global in nature, affecting iscsi initiator behavior on all volumes. In order to avoid unnecessary iscsi logins during system restart, Dell recommends that you change this global setting to manual. For specific volumes that require automatic login, the node.startup property can be set to automatic using the following syntax: # iscsiadm m node T <iqn> -I iface o update n node.startup v automatic Additional information on techniques for managing iscsi logins can be found in Host Integration Tools for Linux Installation and Users Guide. A link to this document is provided in the Revision Information section of this technical report. Load Balancing Policy HIT for Linux MPIO software supports these load balancing policies: Least Queue Depth: Data is sent out to each available connection, with preference given to the connection which is least busy at the time the I/O is requested (RHEL 6). Round Robin: Data is sent out to each available connection in a rotating sequence, effectively utilizing all available ports fully (RHEL 5.5 and RHEL 6). The default load balancing policy is Least Queue Depth, which has the advantage of being bandwidth and latency sensitive. Users can view their current MPIO settings, including the load balancing policy, using the rswcli P command. In most cases, it is best practice to accept the default load balancing policy because it is optimized to run with EqualLogic PS Series arrays. Note that while RHEL 6 supports both load balancing policies, RHEL 5.5 systems only implement the Round Robin policy. Attempting to set the load balancing policy to LQD on a RHEL 5.5 system will result in an error: Maximum Number of Sessions per Volume The number of connections per volume depends on the number of arrays in the group and the number of NIC s being used for MPIO. The two controlling settings for iscsi target connections are: 22

32 Max number of sessions per volume slice Max number of sessions per entire volume Volume slice refers to the slice (or piece) of the volume, which resides on the member storage array. By default, the maximum number of sessions per volume slice is set to 2 and the maximum number of sessions per entire volume is 6. The actual values are based on the number of arrays in the storage pool in which the volume resides: 2 sessions per volume slice * 1 members = 2 sessions per entire volume 2 sessions per volume slice * 2 members = 4 sessions per entire volume 2 sessions per volume slice * 3 members = 6 sessions per entire volume In most cases, it is best practice to use the default settings for both the maximum number of sessions per volume slice and the maximum number of sessions per entire volume. The current maximum session settings can be viewed and modified with the rswcli utility. The rswcli P command will display the current MPIO parameter settings: To increase the maximum number of sessions, use the following syntax: # rswcli P maxvols 3 maxvole 9 This command increases the sessions per volume slice to 3 (default is 2) and the maximum number of sessions per entire volume to 9 (default is 6). For more information about tuning MPIO parameter settings, refer to the Host Integration Tools for Linux Installation and Users Guide listed in the Revision Information section of this technical report. NOTE: Changing MPIO parameter settings may increase the number of simultaneous connections in the configuration. To ensure that you don t exceed available SAN resources, it is important to understand the connection limits for the storage pool prior to changing default MPIO parameters. Configuration File The Dell EqualLogic HIT for Linux software includes a configuration file, stored in plain text format. The file is located here: 23

33 /etc/equallogic/eql.conf The rswcli command provides a convenient and robust way for users to customize many of the settings in this configuration file. Users can also directly edit the configuration file, but the EqualLogic utilities (ehcmcli and rswcli) should always be used to confirm the new settings. Any configuration settings that were customized for a system (e.g. MPIO subnet exclusion) will be preserved across HIT for Linux software upgrades. Summary The Dell EqualLogic PS Series storage array supports multiple iscsi SAN connections for performance and reliability. MPIO provides multiple paths from servers to storage, delivering high availability and improved performance. The Dell EqualLogic HIT for Linux software simplifies MPIO setup and configuration by eliminating the multipath configuration steps and automatically managing multiple iscsi sessions. The software also optimizes the operation of a Dell EqualLogic PS Series array by using knowledge about volume layouts on a PS Series group and routing I/O directly to the Dell EqualLogic PS Series arrays that will be servicing requests. The Host Integration Tools and documentation can be downloaded by logging onto the support website at The tools and documentation are also provided on the CD-ROM that is shipped with the Dell EqualLogic PS Series array. 24

34 TECHNICAL SUPPORT AND CUSTOMER SERVICE Dell s support service is available to answer your questions about PS Series SAN arrays. If you have an Express Service Code, have it ready when you call. The code helps Dell s automated support telephone system direct your call more efficiently. Contacting Dell Dell provides several online and telephone-based support and service options. Availability varies by country and product, and some services might not be available in your area. For customers in the United States, call Note: If you do not have access to an Internet connection, contact information is printed on your invoice, packing slip, bill, or Dell product catalog. Use the following procedure to contact Dell for sales, technical support, or customer service issues: 1. Visit support.dell.com or the Dell support URL specified in information provided with the Dell product. a. Select your locale. Use the locale menu or click on the link that specifies your country or region. 2. Select the required service. Click the Contact Us link, or select the Dell support service from the list of services provided. 3. Choose your preferred method of contacting Dell support, such as or telephone. Online Services You can learn about Dell products and services using the following procedure: 1. Visit (or the URL specified in any Dell product information). 2. Use the locale menu or click on the link that specifies your country or region. 25