PAN Manager SAN Integration Guide

Transcription

1 PAN Manager SAN Integration Guide For BladeFrame BF400 S2 and BladeFrame BF200 Document Number 430-SB0074 August 2010 PM5.2_BF

2 Copyright Copyright 2010 Egenera, Inc. All rights reserved. This document, and the product described in it, is furnished under license and may only be used in accordance with the terms of such license. The content of this document is furnished for information purposes only and is subject to change without notice. Egenera, Egenera stylized logos, BladeFrame, BladeLatch, BladeMate, BladePlane, cblade, Control Blade, PAN Manager, pblade, Processing Blade, sblade, and Switch Blade are either registered trademarks or trademarks of Egenera, Inc. in the United States and/or other countries. PRIMERGY is a registered trademark of Fujitsu Technology Solutions. AMD, AMD Opteron, and AMD Athlon are trademarks of Advanced Micro Devices, Inc.. EMC, CLARiiON, and Symmetrix are registered trademarks of EMC Corporation. The IBM Developer Kit for Linux, Java 2 Technology Edition contains software which is copyright IBM Corporation, Sun Microsystems Inc., Hewlett-Packard Co., and X Consortium. Intel and Itanium are registered trademarks and Xeon is a trademark of the Intel Corporation in the United States and other countries. Sun, Sun Microsystems, the Sun Logo, Solaris, and the Java logo are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. Linux is a registered trademark of Linus Torvalds. Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The virtual VGA console uses Microsoft Terminal Services Advanced Client (TSAC), which is a copyright of Microsoft Corporation. MindTerm is copyright AppGate AB. Nero is a trademark of Nero AG. NetApp is a registered trademark and Network Appliance is a trademark of Network Appliance, Inc. Oracle9i is a trademark of Oracle Corporation. Red Hat is a registered trademark of Red Hat, Inc. in the United States and other countries. SUSE is a registered trademark of SUSE LINUX Products GmbH, a Novell business. VMware, Virtual SMP, and VMotion are registered trademarks or trademarks of VMware, Inc. Xen, XenSource, XenServer, and XenEnterprise are either registered trademarks or trademarks of Citrix Systems, Inc. in the United States and/or other countries. All other company and product names are trademarks or registered trademarks of their respective holders. Printed in the United States of America.[100715:112658] Egenera, Inc., 165 Forest Street, Marlboro, Massachusetts

3 Contents Preface Customer Support... ix Document Conventions... xi Chapter 1: Understanding the PAN and SAN Environments Designing for No Single Point of Failure Supported SAN Ports (HBAs) Storage Connection Types Fibre Channel Topologies Multipath on cblades and pservers Supported I/O Pathing Models for Disks Configuring SCSI Tape Devices in the SAN Chapter 2: Integrating with a SAN Connecting a Platform to the SAN Zoning and Scalability Switched Fabric Zoning Suggestions Planning for Growth iii

4 PAN Manager SAN Integration Guide Managing the Generic Multipath Software Managing SCSI Device Numbering About Device Numbering About Path Device Names About PAN Manager Device Names Controlling SAN Device Names Reasons for Renaming SAN Devices Guidelines for Renaming SAN Devices Rename SAN Devices Maintaining SAN Device Names About SCSI Reservations About SCSI-3 Persistent Reservations About SCSI-2 Reserve/Release Reservations Support for SCSI Tape Resources Best Practices for Configuring SCSI Tape Resources Technical Considerations When Using Tape Devices Chapter 3: Testing your SAN Configuration Validate the Physical Connectivity for SCSI Devices Check Physical Connectivity to SCSI Disk Devices Check Physical Connectivity to SCSI Tape Devices Test I/O on SCSI Devices Test I/O on Each Disk Device Test I/O on Each Tape Device Manage SAN I/O on the cblades Check the Load Balancing on the cblades Check I/O Statistics on Both cblades Take a Snapshot of the Finished SAN Configuration Detect and Correct I/O Imbalances Detect an I/O Imbalance iv

5 Contents Correct an I/O Imbalance Detecting a Possible Zoning Issue Appendixes Appendix A: Troubleshooting and Maintenance SAN Configuration Issues... A-2 Troubleshooting... A-2 Recoverability... A-3 Maintenance Mode... A-4 Diagnosing and Correcting Performance Issues... A-5 Diagnostic Tools... A-5 SAN Failover... A-6 Conditions That Disrupt All SAN Connectivity... A-7 Connection Reinitialization... A-8 Connection Failback... A-8 Appendix B: Using SAN Diagnostic Tools Monitoring I/O Paths with the giscsimon Daemon... B-2 About the giscsimon Daemon... B-2 Examining Log Messages... B-4 Displaying Statistics for I/O Paths... B-6 Editing the giscsimon.conf File... B-9 Using the Start/Stop Script... B-12 Enabling and Disabling HBA Ports with the pathctl Utility... B-12 About the pathctl Utility... B-12 pathctl Utility Arguments... B-13 v

6 PAN Manager SAN Integration Guide Displaying the Current HBA Configuration... B-14 Disabling an HBA Port... B-15 Enabling an HBA Port... B-15 Trespassing a Path... B-16 Displaying LUN Performance Statistics with the pathstat Utility... B-18 About the pathstat Utility... B-18 pathstat Utility Options... B-19 Wildcards in LUN Names... B-21 Columns in pathstat Reports... B-22 pathstat Multipath Report (Default)... B-23 pathstat Multipath Report for Specific LUNs... B-25 pathstat Subpath Report... B-25 pathstat Topology Report... B-26 Displaying HBA Port Performance Statistics with the pathtop Utility... B-27 About the pathtop Utility... B-27 pathtop Utility Defaults... B-28 pathtop Statistics Format... B-29 pathtop Utility Options... B-30 pathtop Example... B-32 Appendix C: SAN Resource Guidelines and Limitations Index vi

7 Preface Welcome to PAN Manager PM5.2_BF. PAN Manager SAN Integration Guide is part of the PAN Manager documentation set. Its purpose is to describe the best practices to employ when integrating a Processing Area Network (PAN) with a data center s storage area network (SAN) environment. Audience PAN Manager SAN Integration Guide is for PAN Administrators and LPAN Administrators. Topics Read this book to learn about the following: Understanding the PAN and SAN Environments Integrating with a SAN Testing your SAN Configuration Troubleshooting and Maintenance Using SAN Diagnostic Tools SAN Resource Guidelines and Limitations Customer Support For customer support regarding this product, please use the following contact information: PM5.2_BF ix

8 PAN Manager SAN Integration Guide Fujitsu Technology Solutions customer support Internet Telephone See the Help Desk information at manuals.ts.fujitsu.com/primergyservers.html, under Contact. Important: Fujitsu Technology Solutions supports only storage devices that it has explicitly certified. Before consulting Fujitsu Technology Solutions customer support, see the Storage Interoperability Matrix at for a list of supported storage devices. x PM5.2_BF

9 Preface Document Conventions Convention Description > Directory-level delimiter used to navigate the left pane of the PAN Manager GUI. Example: Resources > Ethernet Connections Sans serif italics Variable text, such as a path, a filename, or an LPAN name. Example: lpan -c lpanname Sans serif Bold Italics [text] {text} Text that must be typed as shown. Example: Type root at the login prompt. The name of a field or window element appearing in a GUI. It also highlights default values in PAN Manager man pages. Example: In the Users page... Text that is emphasized. Example: Do not connect the power. Text that is optional to a command. A set of choices, one of which is required. Separation of mutually exclusive choices in syntax lines. Example: lpan [-ad -rd]{switch SCSI_ID} lpanname Note Caution! Information of importance or that may not fit in main text. Failure to heed a caution could result in loss of data. Warning Failure to heed a warning could result in physical harm to the user or the hardware. PM5.2_BF xi

10 PAN Manager SAN Integration Guide xii PM5.2_BF

11 Beta Chapter 1 Understanding the PAN and SAN Environments This chapter describes various SAN features and how to configure them. Designing for No Single Point of Failure Supported SAN Ports (HBAs) Storage Connection Types Multipath on cblades and pservers Supported I/O Pathing Models for Disks Configuring SCSI Tape Devices in the SAN PM5.2_BF 1-1

12 PAN Manager SAN Integration Guide Designing for No Single Point of Failure Beta The ultimate goal when configuring a SAN, and integrating the platform and its PAN into the SAN, is to create sufficient redundancy that the failure of a single component will not interrupt connectivity between servers and storage. The components of an integrated PAN and SAN are as follows: A cblade A SAN switch A control port on a storage array Fibre Channel cables This goal of designing the SAN for no single point of failure is met by: the inherent redundancies in the physical architecture of the PAN and its virtualization software the use of multipathing software for pservers when accessing storage resources configuring the SAN with redundancy in mind Figure 1.1 on page 1-3 shows how the PAN and Fibre Channel SAN environments interact, and suggests a configuration in which a component failure would not interrupt connectivity between 1-2 PM5.2_BF

13 Beta About SAN Components of the PAN Figure 1.1 Fibre Channel SAN: A Multiple-Switch Fabric Supported SAN Ports (HBAs) The following table lists the SAN ports (host bus adapters) available on the cblades. PM5.2_BF 1-3

14 PAN Manager SAN Integration Guide Beta Table 1.1 HBA Configurations, by Platform Platform Blade Type Port Numbers Total Ports per System BladeFrame BF400 S2 cblade EX SCSI 4, 3, 2, 1 8 BladeFrame BF200 cblade ES SCSI 3, 4 4 Figure 1.2 shows the location of the Fibre Channel ports on the front of a single cblade EX. Figure 1.3 shows the location of the Fibre Channel ports on the back of a BladeFrame BF200 chassis. Figure 1.2 BladeFrame BF400 S2: HBA Locations on a cblade SAN ports: SCSI 4 SCSI 3 SCSI 2 SCSI 1 Figure 1.3 BladeFrame BF200: HBA Locations on each cblade SAN ports: SCSI 3 SCSI 4 SCSI 3 SCSI PM5.2_BF

15 Beta Storage Connection Types About SAN Components of the PAN This section provides information about the following SAN topologies: Fibre Channel Topologies Refer to the appropriate section, according to the type of SAN ports provided on the platform s cblades. Fibre Channel Topologies A platform can connect to a Fibre Channel SAN environment in one of three ways: Through a direct and arbitrated loop connection Through a single-switch fabric connection Through a multi-fabric connection The following sections provide information about these connection types: Direct Attach and Arbitrated Loop Configurations Using Fabric Switches in a Fabric Single-Fabric Configuration Multi-Fabric Configuration Direct Attach and Arbitrated Loop Configurations The simplest connection types are the direct attach and arbitrated loop from the platform to one, or more, storage arrays (see Figure 1.4 on page 1-6). The disadvantage to the direct attach connection type is that you are limited by the number of HBA ports available on the platform. You can only connect to the same number of SAN storage arrays. PM5.2_BF 1-5

16 PAN Manager SAN Integration Guide Beta In this connection type, each cblade HBA connects (through physical cables) directly to the storage devices. This storage connection type allows the HBAs on each cblade to see the same storage device. Figure 1.4 Direct Attach Connections cblade 2 Storage Array cblade 1 Storage Array Using Fabric Switches in a Fabric Fibre Channel networks built with switches are called fabrics. Using fabric switches provides multiple ways to route traffic from an HBA port to a target array on the SAN. (See Figure 1.1 on page 1-3.) A limitation to connecting to one fabric switch (instead of multiple fabric switches): if the switch fails, access to the data is lost. In a properly configured multi-switch fabric, there is no single point of failure: if one switch fails, access to the data is maintained through an alternate path over one of the other switches. Single-Fabric Configuration A platform supports direct connection through a single fabric switch (Figure 1.5). 1-6 PM5.2_BF

17 Beta About SAN Components of the PAN Figure 1.5 A Single Fabric Switch Configuration Each cblade HBA connects to the same fabric switch. This storage connection type allows the adapters on each cblade to see the same storage devices that connect to the fabric. PM5.2_BF 1-7

18 PAN Manager SAN Integration Guide Beta Figure 1.6 A Multiple-Switch Fabric Multi-Fabric Configuration A platform also supports connection through two or more fabrics (see Figure 1.6 on page 1-8). In the example in this figure, the following are true: HBAx from cblade A and cblade B connects to fabric switch 1 HBAy from cblade A and cblade B connects to fabric switch PM5.2_BF

19 Beta About SAN Components of the PAN This storage connection type allows the two HBAx initiator ports (one on each cblade) to see the same storage devices that connect to the fabric. The same is true for both HBAy initiator ports for the other fabric. The storage devices themselves connect to both fabrics. Multipath on cblades and pservers This section describes the features available with generic multipath support: How multipath choices influence which connection type to use How a SAN disk array s support of certain modes of operation (active/active, active/passive, or preferred path) affects which multipath option to use Multipath support is provided for paths to SAN tape devices. Refer to Configuring SCSI Tape Devices in the SAN on page 1-13 for more information. PM5.2_BF 1-9

20 PAN Manager SAN Integration Guide Beta Multipath on the pservers Multipath on the pservers is enabled by default, and provides a path for I/O requests to be sent through either cblade to external SAN disk storage. Multipath on pservers allows pservers to continue operation even if one cblade fails or reboots. Figure 1.7 illustrates multipath from a pserver to the cblades. Figure 1.7 Multipath From a pserver to the cblades cblade 2 pserver Multipath from pserver cblade 1 For example, if cblade 1 fails, multipath allows I/O requests from the pserver to be retried on cblade PM5.2_BF

21 Beta About SAN Components of the PAN Multipath on the cblades Figure 1.8 depicts multipath from the cblades to external storage. Figure 1.8 Multipath From cblades to External Storage Multipath from pserver HBAx cblade 2 SAN Switch 1 pserver Multipath from cblade HBAy HBAy Storage Array cblade 1 HBAx SAN Switch 2 This figure shows one possible configuration type (a multi-switch topology) that supports multipath. The host bus adapters on each cblade connect to the storage environment through SAN switches, so that the HBAx and HBAy adapters on each cblade see the same disk storage. Multipath distributes the I/O requests among the four paths from cblade 1 and cblade 2 to SAN storage. This increases the bandwidth of the I/O and the number of I/O operations per second. Also, if one cblade fails or reboots, the other can fulfill any I/O requests. PAN Manager assigns a name to each multipath device on the cblades during the initial discovery process. This discovery takes place when the driver loads on the cblade at boot time. Adding or removing storage arrays could change the discovery order of the target controllers. (See Managing SCSI Device Numbering on page 2-5 about the impact that adding or removing storage arrays can have on SCSI device numbering.) PM5.2_BF 1-11

22 PAN Manager SAN Integration Guide Supported I/O Pathing Models for Disks Beta Pathing models determine how a storage array presents its disk LUNs to the network. The PAN environment supports the following I/O pathing models for SAN disk resources: Active/active Active/passive Primary/secondary or Preferred path In an optimal situation, I/O uses active and primary paths, instead of passive or secondary paths. Passive and secondary paths remain available to assume I/O in suboptimal situations. Active/active Allows a LUN to be visible on all ports of the storage array. This is a fully symmetric pathing model in which all paths are available for I/O to the disk LUN. In this model, if an active path fails, multipath retries the I/O on the other active path. Multipathing software can be very useful to manage the complexities that this model allows. For information about multipathing software, see PAN Manager Technical Overview. Active/passive Multipath tries I/O on all active paths on both cblades. If I/O fails on an active path, multipath activates the corresponding passive path (a process sometimes called trespassing), and then retries I/O on the newly active path. (Multipath must issue specific commands in order to change a passive path to an active one.) For information about active and passive paths, see Detect and Correct I/O Imbalances on page Primary/secondary or Preferred path Multipath tries I/O on all optimal (primary) paths on both cblades. If I/O fails on a primary path, multipath retries the I/O on a secondary path PM5.2_BF

23 Beta About SAN Components of the PAN A primary path offers faster access to a disk LUN. A secondary path typically offers a slower path to a disk LUN, and is used as a standby path. Note: Pathing models do not apply to SAN tape devices because each SAN tape device operates through a single I/O path. Configuring SCSI Tape Devices in the SAN Supported tape devices (including tape drives and any tape libraries) can connect directly to a fabric switch, typically a core switch. Because connecting a tape drive directly to an HBA on the platform can diminish the bandwidth available to disk I/O, we do not recommend connecting a tape device directly to the platform. Figure 1.9 shows a typical SAN configuration for tape resources. PM5.2_BF 1-13

24 PAN Manager SAN Integration Guide Beta Figure 1.9 Tape Devices in a SAN Environment Not all pserver releases support SCSI tape devices. For more information about which pserver software supports tape devices, see PAN Manager Release Notes. In PAN Manager, the target IDs for SCSI tape devices range from 224 to 255; the LUN number of each SCSI tape device is zero. For SCSI tape devices, as with all fabric devices, use singleinitiator zoning. Each tape device must be zoned to a single HBA on a single cblade. For more details about zoning, see Switched Fabric Zoning Suggestions on page PM5.2_BF

25 Beta About SAN Components of the PAN Multipath software is not supported for Fibre Channel tape devices. Because a pserver accesses each SCSI tape device through only one cblade, automatic load balancing and failover for SCSI tape devices is not supported. Testing your SAN Configuration on page 3-1 describes how to use cblade utilities to detect failures and to balance I/O manually across HBA ports. For more information and guidelines about configuring SCSI tape devices for use by pservers, see Best Practices for Configuring SCSI Tape Resources on page PM5.2_BF 1-15

26 PAN Manager SAN Integration Guide Beta 1-16 PM5.2_BF

27 Beta Chapter 2 Integrating with a SAN This chapter describes how to integrate one or more platforms into a SAN environment. Connecting a Platform to the SAN Zoning and Scalability Managing the Generic Multipath Software Managing SCSI Device Numbering Controlling SAN Device Names About SCSI Reservations Support for SCSI Tape Resources Connecting a Platform to the SAN Important: Your SAN administrator should have already attached the SAN cables to the hardware platform s cblades. If the SAN cables are not connected to the cblades, see READ THIS FIRST: PAN Manager Configuration Guide for information about the cblade SCSI ports. PM5.2_BF 2-1

28 PAN Manager SAN Integration Guide Zoning and Scalability Beta This section describes the best practices and recommended configurations to use when creating SAN zones, and provides suggestions on how to ensure that the integration supports flexible and scalable configurations. The purpose of zoning is to isolate storage resources for use by some servers, and to exclude the use of these storage resources from use by other servers. Zoning is configured in the switch director software on the fabric switches in your SAN. The following sections provide the background and guidelines for configuring SAN zoning: Switched Fabric Zoning Suggestions Planning for Growth For details about troubleshooting zoning issues, see Detecting a Possible Zoning Issue on page Switched Fabric Zoning Suggestions Storage administrators frequently segment their switched fabric environments into soft or hard zones. This limits access to certain ports, and distributes I/O traffic among various controllers in the SAN. Single Initiator Zoning We recommend that the SAN administrator implement singleinitiator-to-single-target zoning. For example, an initiator port with a unique worldwide name (WWN) would have a single target with a different WWN. In general, single-initiator-to-single-target zoning is a good isolation mechanism and can better withstand fabric changes than other types of zoning. 2-2 PM5.2_BF

29 Beta Integrating with a SAN Hard Zoning Hard zones provide the highest security and help prevent unauthorized entry into a zone. In this zoning model, an initiator cannot see outside the zone(s) to which it belongs. In multiswitched environments, it is best to zone to segregate storage I/O traffic between groups of servers and their storage subsystems and devices. Planning for Growth The platform integrates seamlessly into SAN environments, and provides scalability through its multipath I/O driver support. As your SAN and enterprise networks grow, the extensible PAN architecture can scale to accommodate new configurations. Managing the Generic Multipath Software When PAN Manager is started or does a disk scan to discover new disks or paths, the cblade software creates the file /etc/ sysconfig/ifc/multipath_devices on each cblade. This file maps the unique SAN device IDs to their respective path devices; the kernel path driver uses the file to reference SAN path devices. We recommend configuring SCSI disk devices symmetrically on the platform(s) in your PAN. This means that each Control Blade in a platform has access to an identical set of SCSI disk devices. The following is an example of the contents of the multipath_devices file: HITACHI-OPEN3-VID-HITACHIR401ED2A0090:(9,0,0,30):(3,0,0,30) HITACHI-OPEN3-VID-HITACHIR401ED2A0091:(9,0,0,31):(3,0,0,31) HITACHI-OPEN3-VID-HITACHIR401ED2A0092:(9,0,0,32):(3,0,0,32) PM5.2_BF 2-3

30 PAN Manager SAN Integration Guide Beta HITACHI-OPEN3-VID-HITACHIR401ED2A0093:(9,0,0,33):(3,0,0,33) HITACHI-OPEN3-VID-HITACHIR401ED2A0097:(9,0,0,34):(3,0,0,34) HITACHI-OPEN3-VID-HITACHIR401ED2A0098:(9,0,0,35):(3,0,0,35) Caution: Because the PAN Manager software and the kernel can independently assign SAN device IDs, we strongly recommend that you save a copy of each cblade multipath_devices file off the platform for safekeeping. If, after adding or removing SAN disks, the kernel s list of device IDs and the PAN Manager s list of device IDs do not match, do the following: 1. Determine the cause of the differences between the device mappings. There may be valid or benign reasons for the differences. 2. If you cannot determine the reason for the difference, copy the archived file back to /etc/sysconfig/ifc, and then reboot the cblades to restore the previous device ID mappings. If PAN Manager determines that the contents of this file changed since the last reboot, it creates a backup (multipath_devices.pmsaved) of the current file before writing the new file and changing its contents. To keep a copy of the last known state of the multipath device mapping, archive this backup file. You can then use the backup copy if you need to recover the information for any reason. 2-4 PM5.2_BF

31 Beta Managing SCSI Device Numbering Integrating with a SAN Under some circumstances, the names of some devices can have an unexpected form, or can change after a cblade reboots. This can occur in the following instances: You change the SAN configuration (including adding or removing one or more disks). You change the kernel s SCSI settings. You change path groups on existing devices, and then reboot (which may change the UIDs of existing devices). These changes can impact device names in two different ways: A device s path device name can change from 9.n.n.n to 9.n.128.n. A device s PAN Manager device name can change from 9.n.n.n to 10.n.n.n. Although this condition is harmless, it may cause some confusion. This section describes how to identify and correct this condition, as desired, and where possible, how to prevent it. About Device Numbering The cblade software (kernel and PAN Manager) names SCSI devices in various ways. The following summarizes these names and how they affect each other. PAN Manager device name Appears in the same format as a SCSI ID (n.n.n.n), and used in PAN Manager for display and configuration. PAN Manager assigns this name to the disk when it discovers the disk for the first time. Path device name The name of the disk as recognized by the cblade kernel and which represents the aggregate paths (subpaths) by which the device is accessible. The multipath driver creates this name each time the kernel boots. PM5.2_BF 2-5

32 PAN Manager SAN Integration Guide Beta In most cases, a device s PAN Manager device name and path device name are identical. However, if there are SAN configuration changes or changes in the kernel SCSI settings, a physical device s path device name can change after a cblade reboots. (It is also possible for one cblade kernel to recognize a physical device by one path device name, and for the other cblade kernel to recognize the same device by a different path device name.) Unique identifier (UID) A long string used by the kernel to identify the device. This UID persists for the lifetime of the device. Because PAN Manager persists the mapping between the UID and the PAN Manager device name, PAN Manager configurations are not affected by any changes in the device s path device name. For example, in PAN Manager, you use the PAN Manager device name to display a device s characteristics: # san -l "( )" In the command output, the PAN Manager device name appears in the ID column, the Unique ID appears on its own line, and the path device names, as recognized by each cblade kernel, appear in the Path Device column: Description: ID Capacity Type S/N Paths LPAN ( ) 20.0GB 3PARdata/VV 000C022B 2 my_lpan Unique ID: 3PARdata-VV-WWN-50002ac0000c022b I/O Paths: Cblade Path Device Subpaths minigolf/c2 ( ) ( ),( ) minigolf/c1 ( ) ( ),( ) Partitions: N/A Assignments: my_lpan (pserver4:(56.0)) 2-6 PM5.2_BF

33 Beta About Path Device Names Integrating with a SAN This section describes how path device names can change, and how to help prevent this from happening. The Cause of Path Device Name Changes When PAN Manager scans and detects a new device, the cblade software establishes two subpaths for it, for example and The cblade software then attempts to construct a single path device name (such as ) that corresponds with each subpath. For any of the reasons listed under Managing SCSI Device Numbering on page 2-5, the cblade software may associate one subpath ( ) with a path device name ( ), but be unable to associate the other subpath ( ) with this same path device name. For example, if a device has these names: PAN Manager device name: ( ) Path device name: ( ) UID: uid-1 and the SAN in which the device resides is reconfigured (a device is added), the new device might have these names: PAN Manager device name: (?.?.?.?) Path device name: ( ) UID: uid-2 Because the path device name of the new device ( ) would be identical to the path device name of an existing device, this causes the path device name of the new device to become n (in this case ): PM5.2_BF 2-7

34 PAN Manager SAN Integration Guide Beta PAN Manager device name: ( ) Path device name: ( ) UID: uid-1 and PAN Manager device name: (?.?.?.?) Path device name: ( ) UID: uid-2 To prevent a device from being recognized by two different path device names, be sure to adhere to the guidelines described in the section Preventing Path Device Name Changes. Preventing Path Device Name Changes To help prevent the situation in which a single device is identified by multiple path device names, check to see that the LUNs you expose are visible on all paths as the same LUN number. About PAN Manager Device Names This section describes how PAN Manager device names can change, how to detect this condition, and if necessary, how to synchronize PAN Manager device names with their associated path device names. The Cause of PAN Manager Device Name Changes When PAN Manager discovers a new device, it attempts to generate a PAN Manager device name that is identical to the new device s path device name. 2-8 PM5.2_BF

35 Beta Integrating with a SAN If the new device s PAN Manager device name is already in use in PAN configurations, PAN Manager generates an easily noticeable PAN Manager device name for the new device, in the form 10.n.n.n, for example This yields the following names for the two devices: PAN Manager device name: ( ) Path device name: ( ) UID: uid-1 and PAN Manager device name: ( ) Path device name: ( ) UID: uid-2 Important: Before importing SCSI devices into the PAN, be sure that each discovered device s PAN Manager device name and path device name are identical. To check that PAN Manager device names and path device names are identical: 1. In PAN Manager, turn the auto import feature off: # san -m 2. Display all devices (those in the PAN and those not in the PAN): # san -A Comparing PAN Manager Device Names and Path Device Names This section describes how to determine whether the PAN Manager device names and path device names of one or more devices are identical, and if they are not, how to make them identical. To compare the PAN Manager device names and path device names for all SAN devices, enter: # san -l PM5.2_BF 2-9

36 PAN Manager SAN Integration Guide Beta Output similar to the following appears for each device: Description: ID Capacity Type S/N Paths LPAN ( ) 20.0GB 3PARdata/VV 000C022B 2 my_lpan Unique ID: 3PARdata-VV-WWN-50002ac0000c022b I/O Paths: Cblade Path Device Subpaths minigolf/c2 ( ) ( ),( ) minigolf/c1 ( ) ( ),( ) Partitions: N/A Assignments: my_lpan (pserver4:(56.0)) In this case, the PAN Manager device name (at top left, in the ID column) is identical to each path device listed in the Path Device column. If there are differences between the path device names and PAN Manager device names of one or more devices in your SAN, follow the procedure Rename SAN Devices on page Controlling SAN Device Names This section describes how to use the PAN Manager option for explicitly renaming SCSI disk and tape devices. Note: Support for Fibre Channel tape devices requires having Fibre Channel HBAs on the platform s cblades. Reasons for Renaming SAN Devices Guidelines for Renaming SAN Devices 2-10 PM5.2_BF

37 Beta Integrating with a SAN Rename SAN Devices Maintaining SAN Device Names PAN Manager provides an optional way to persist a SAN device s SCSI ID as presented by PAN Manager, despite configuration changes in the SAN. Although there is no technical requirement for doing this, some administrators may find this feature useful for maintaining a consistent device naming policy in the PAN. This is particularly useful when managing PAN archives (disaster recovery files) across multiple hardware platforms. Important: Rebooting the cblades will not change any SCSI IDs unless there have been changes in the SAN s configuration (devices added or removed). Reasons for Renaming SAN Devices Renaming SAN devices is useful for sites that: Use the disaster recovery (DR) feature to create PAN archives for the purpose of replicating PAN configurations on other hardware platforms. For example, if a particular pserver's disk device is named on the production hardware platform, but is named on the DR hardware platform, you can rename one or the other device names so that they match. This can help prevent confusion during a PAN DR swap operation and simplify management of your hardware platforms. (It makes PAN archives easier to use during the "map" phase if the SCSI IDs are the same on both platforms.) Want the PAN Manager device names (SCSI IDs) for SAN devices to remain the same, despite changes to the PAN s overall SAN configuration. Want PAN Manager SCSI IDs of devices newly added to the PAN to conform to the site s device naming conventions. PM5.2_BF 2-11

38 PAN Manager SAN Integration Guide Beta Guidelines for Renaming SAN Devices This section describes under what conditions you can rename SAN devices. In order to rename a SAN device: The device must already have been imported to the PAN. The device must not be allocated either to an LPAN or configured for use in disaster recovery. You must use the PAN Manager command line interface (CLI). (Renaming SAN devices is not supported in either the PAN Manager GUI or the PAN API.) The new name you select for the device must: not already be used by a SAN device in the PAN follow the PAN Manager 4-tuple SCSI ID naming convention (w.x.y.z) fall in the acceptable range for SCSI IDs, where w is from 0 to 255 x is from 0 to 255 y is from 0 to 511 z is from 0 to 4095 Rename SAN Devices This procedure describes how to change any devices in the PAN not to have PAN Manager device names of the form 10.n.n.n. Consider performing this procedure if one or more reasons for doing so applies to your site; see Reasons for Renaming SAN Devices on page To change one or more devices in the PAN not to have PAN Manager device names of the form 10.n.n.n: 1. If you are renaming one or more SAN devices that are already configured in an LPAN/pServer, remove each device from its LPAN/pServer or from its configuration as a DR device PM5.2_BF

39 Beta Integrating with a SAN 2. Capture the SCSI IDs in your current SAN configuration, (and the kernel Unique ID (UID) to which each SAN device corresponds). # san -l egrep -A 3 '^\ID' > anyfile Output similar to the following appears: ID Capacity Type S/N Paths LPAN ( ) 7MB EMC/SYMMETRIX Unique ID: EMC-SYMMETRIX-WWN d ID Capacity Type S/N Paths LPAN ( ) 8.4GB EMC/SYMMETRIX Unique ID: EMC-SYMMETRIX-WWN d You will use this information in Step 4 when renaming your SAN devices. 3. PAN Manager does not support persistent bindings. You should have removed any persistent bindings before upgrading PAN Manager to Release 5.2.n. If you are not sure whether you had persistent bindings configured before upgrading PAN Manager to Release 5.2.n, open the /crash_dumps/backup_data_files/etc/modules.conf file. (The PAN Manager upgrade process archived this file to this location from /etc/modules.conf.) If there are any lines in modules.conf that are similar to the following: scsi-qla0-tgt-10-di-0-port=50060e ;\ scsi-qla0-tgt-11-di-0-port= d12a;\ then one or more of your SAN devices were configured to have persistent bindings. Because the SCSI IDs of these devices can change if there is a change to your overall SAN configuration, you must now set a specific SCSI ID for each device. To do so, go on to Step For each SAN device, specify the following: # san -n (new_scsi_id) { (SCSI_ID) UID } PM5.2_BF 2-13

40 PAN Manager SAN Integration Guide Beta where new_scsi_id is the SAN device s new PAN Manager SCSI ID SCSI_ID is the SAN device s current PAN Manager SCSI ID UID is the SAN device s Unique Identifier: a long string used by the kernel to identify the SAN device. (This string never changes.) When renaming the device s SCSI ID, you can either specify its current SCSI ID, or its UID. For example, to rename the device to , enter: # san -n ( ) ( ) To specify a new SCSI ID for a device whose UID is EMC- SYMMETRIX-WWN d323741, enter: # san -n ( ) EMC-SYMMETRIX-WWN d If you have renamed any SAN devices that were configured in an LPAN/pServer or configured as a DR device, return each device to its original configuration. Maintaining SAN Device Names To maintain naming control of your SAN devices, perform the procedure in Rename SAN Devices on page 2-12 on an ongoing basis, at the following times: Immediately after the first cblade reboot following a SAN configuration change Immediately after performing a manual disk scan (san -s) of the SAN following a SAN configuration change PM5.2_BF

41 Beta About SCSI Reservations Integrating with a SAN The cblade software provides support for two types of SCSI reservations: A persistent reservation As defined by the SCSI-3 standard, is an association between a pserver running on any Processing Blade and a SAN disk, and allows more than one booted pserver to simultaneously access the same disk. For the persistent reservation feature, PAN Manager lists both the pserver registrations and reservations associated with the disk. A reserve/release reservation As defined by the SCSI-2 standard, allows only a single booted pserver to access a single device. By default, PAN Manager enables all devices with reserve/release reservations. Some applications that you install on pservers may require SCSI reservations. For instructions for enabling and disabling persistent reservations, and for clearing stale SCSI-2 or SCSI-3 reservation data from disks, see PAN Manager Administrator s Guide. To use the persistent reservation feature on any disk, you must use the array vendor s command set to enable persistent reservations for that disk. For details about enabling persistent reservations, refer to your array vendor s documentation. About SCSI-3 Persistent Reservations For Linux pservers, the SCSI-3 persistent reservation feature allows more than one booted pserver to share the same data disk (LUN) in a controlled fashion, primarily for clustering applications. Typically, a persistent reservation is an association between the ports of a server and the ports of an array. In the PAN environment, a persistent reservation is actually an association between a pserver running on any pblade and a SAN disk device. The persistent reservation is used to control access between the pservers and the disk over any of the configured paths between the cblade HBA ports (initiator ports) and the SAN ports (target ports). PM5.2_BF 2-15

42 PAN Manager SAN Integration Guide Beta About SCSI-2 Reserve/Release Reservations SCSI-2 (reserve/release) reservations prevent unwarranted access to a pserver s SCSI devices by other pservers. SCSI-2 reservations govern how pservers can or cannot access the same disk LUN. The cblade enforces the reservation with respect to I/O from other pservers. The cblade does not forward reserve (and release) commands to the device itself. This allows the cblade to continue to loadbalance among all the paths to the device even when the device is reserved. This results in better performance from some storage arrays in the presence of a SCSI-2 reservation. Support for SCSI Tape Resources Note: Support for Fibre Channel tape devices requires having Fibre Channel HBAs on the platform s cblades. Best Practices for Configuring SCSI Tape Resources This section describes various ways to optimize your use of tape backup resources in a PAN environment: Distribute tape drives across the cblades. This means that if your SAN has multiple zones, try to distribute the tape devices evenly across these zones. For security reasons, it is desirable for each SCSI tape device in the PAN to have one hard zone path. For more information about zoning, see Zoning and Scalability on page 2-2. Configure tapes on one HBA on a single cblade. For performance reasons, it is best to configure all SCSI tape devices to use as few cblade HBAs as possible, without exceeding the I/O capacity of individual HBAs. Use the pathtop and pathstat utilities to determine the I/O performance of specific SCSI devices and specific HBAs. For additional information about using these utilities, see Chapter 3, Testing your SAN Configuration PM5.2_BF

43 Beta Integrating with a SAN It is possible to configure a disk device and a tape device on the same HBA, but only one of these devices can perform I/O operations at any given moment. In general, if a disk is using an HBA, and a tape device that is configured on that same HBA starts processing I/O, all disk I/O on the disk ceases. (This behavior helps maximize the tape device s throughput.) An exception is when the shared HBA is the disk s only functioning device path. In that case, disk I/O and tape I/O can occur simultaneously on the same HBA. When perfoming tape backup operations, you can use bandwidth best by ensuring that the disk you are reading from, and the tape you are writing to, are configured on different pservers. If you have multiple pservers that will act as SCSI tape device media servers on a platform, you can optimize performance by ensuring that their pblades are evenly divided between odd and even slots in the platform. For example, if you have four pservers that will be media servers, select two even numbered pblades (such as p2 and p6), and two odd numbered pblades (such as p3 and p5). You can assign a SCSI tape device to multiple pservers within an LPAN; however, be sure that the applications installed on these pservers use reserve/release (SCSI-2) reservations to prevent unwarranted access to a pserver s SCSI tape device(s) by another pserver. For more information about reserve/release reservations for SCSI disk and tape device resources, see About SCSI Reservations on page Following these practices will help optimize the performance and reliability of your data backups onto tape. Technical Considerations When Using Tape Devices If you are using tape devices, keep in mind the following: Multipath support is not provided for paths to SAN tape devices, because a SCSI tape device uses only one path. The pservers of an LPAN can share a specific SCSI tape device. A SCSI tape device cannot be shared across LPANs. PM5.2_BF 2-17

44 PAN Manager SAN Integration Guide Beta Caution: The Linux tape class driver gets its maximum transfer size from the SCSI tape device. If the maximum transfer size for a tape device is greater than 128 KB, the I/O operation fails. To avoid this issue when running applications, use a transfer size of 128 KB or smaller PM5.2_BF

45 Beta Chapter 3 Testing your SAN Configuration This chapter describes how to test the combined PAN and SAN environments that you have configured. Although these procedures are of greatest value when you first configure your SAN, you can also use them to test your SAN connectivity at any time in an existing environment. Various factors can affect the I/O behavior you may see when you perform the procedures in this chapter. If you are experiencing uneven results, capture any applicable utility output, and contact Fujitsu Technology Solutions customer support or your authorized support vendor for assistance. This chapter includes the following tests: Validate the Physical Connectivity for SCSI Devices Test I/O on SCSI Devices Manage SAN I/O on the cblades Take a Snapshot of the Finished SAN Configuration Detect and Correct I/O Imbalances PM5.2_BF 3-1

46 PAN Manager SAN Integration Guide Beta The examples provided in this chapter are limited in scope by the following factors: The examples assume that you are configuring Linux pservers. (Your own pserver operating system may be different.) The examples assume that you are running the generic multipath software rather than third-party multipath software. The examples are valid for SCSI IDs (LUNs) that are accessed through active/active controllers. (Output results differ if your pservers access LUNs through active/passive controllers or preferred path controllers.) The examples assume that you are using the PAN Manager GUI to determine the SCSI IDs (and subpaths) of SCSI devices. (You can also use the PAN Manager CLI for this purpose.) Validate the Physical Connectivity for SCSI Devices The goal of the following two procedures is to determine whether you have physically configured the SAN arrays, fabric switches, and cables correctly. Check Physical Connectivity to SCSI Disk Devices Perform this procedure for each SCSI disk device in the PAN. To check physical connectivity to SCSI disk devices: 1. Using a web browser, log on to the PAN Manager GUI. 2. In the PAN Manager GUI left pane, select Resources > Disks. The SCSI Disks page displays all SCSI disk devices in the PAN. 3. Select a disk by clicking its SCSI ID in the ID column. 4. On the disk_name page, in the Type field, be sure that the Vendor and Model of the disk s storage array are accurate. 3-2 PM5.2_BF

47 Beta Testing your SAN Configuration 5. On this same GUI page, in the IO Paths area, click the icon for each cblade. The information that appears includes the subpaths associated with this disk. a. Be sure that there is a subpath for each HBA configured on your cblades. (The first number in a subpath s SCSI ID represents the HBA number on the cblade. For example, the subpath indicates that I/O passes through HBA 3.). b. Be sure that both cblades display the same subpaths and SCSI ID for the disk, indicating that the device is accessible to both cblades. If the number of subpaths you see is more than expected, the disk s unique ID may not be unique. If the number of subpaths you see is less than expected: There may be a cabling issue. There may be an issue with an array controller. Zoning may be improperly configured. If the number of subpaths continues to be more or less than you expect, contact Fujitsu Technology Solutions customer support or your authorized support vendor for assistance. Check Physical Connectivity to SCSI Tape Devices Perform this procedure for each SCSI tape device in the PAN. Note: Support for Fibre Channel tape devices requires having Fibre Channel HBAs on the platform s cblades. To check physical connectivity to SCSI tape devices: 1. Using a web browser, log on to the PAN Manager GUI. 2. In the PAN Manager GUI left pane, select Resources > Tapes. The SCSI Tapes page displays all SCSI tape devices in the PAN. 3. Select a tape device by clicking its SCSI ID in the ID column. 4. On the SCSI Tape tape_name page, in the Type field, be sure that the Vendor and Model of the tape device are accurate. PM5.2_BF 3-3

48 PAN Manager SAN Integration Guide Beta 5. On this same GUI page, in the IO Paths area, click the icon for each cblade. The information that appears includes the subpath associated with this tape device. There should be only one subpath. The first number in the subpath s SCSI ID represents the HBA number on the cblade through which the tape device is connected. If more than one subpath appears: The SCSI tape device s unique ID may not be unique. Zoning may be improperly configured. If this behavior persists, contact Fujitsu Technology Solutions customer support or your authorized support vendor for assistance. Test I/O on SCSI Devices The goal of this procedure is to test sample I/O traffic between pservers and the storage devices configured on them. The procedure includes creating one or more pservers for the sole purpose of testing all storage devices in the PAN. For information about creating an LPAN and pservers, see PAN Manager Administrator s Guide. Test I/O on Each Disk Device To ensure the flow of I/O to and from a specific SCSI disk device: 1. Using a web browser, log on to the PAN Manager GUI. 2. Create an LPAN, and then create one or more pservers in the LPAN. (See PAN Manager Administrator s Guide, if necessary.). 3. Add the same number of SCSI disk devices to each pserver that you created. 3-4 PM5.2_BF

49 Beta Testing your SAN Configuration 4. Install the same pserver software on each pserver. (This example uses the Linux operating system. Refer to the appropriate pserver Guide for instructions for installing the pserver software.) 5. Install an application or use an operating system utility that generates I/O. (In this case, we use the Linux dd command.) 6. Use the network utility of your choice (such as ssh or PuTTY) to open a remote network console on each cblade. 7. Display all disks assigned to a specific pserver. To do this, in the PAN Manager GUI left pane, select LPANs > lpan_name > pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. 8. Open a console on the pserver, by doing one of the following: Using a network utility of your choice (such as ssh or putty), log on to one of the cblades, and then enter console lpan_name/pserver_name where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. In the PAN Manager GUI left pane, select LPANs > lpan_name > pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. On the pserver pserver_name page, in the bottom border, click the console icon. 9. On the pserver pserver_name page, in the Current Blade area, determine the name of the pblade that is configured on the pserver. The pblade name appears as a link, in the format platform/pn, where N is the slot number of the pblade in the platform chassis. 10. To use this pblade s slot number to display I/O statistics for this pserver, issue this command, as appropriate, for your hardware platform, in both cblade consoles: # pathstat -T p3 -u -t -i 1 This example displays a topology report at one-second intervals for the pblade in Slot 3, and shows that this pblade is configured on the pserver mylpan/ps5. PM5.2_BF 3-5

50 PAN Manager SAN Integration Guide Beta Thu Feb 2 15:09: pblade topology pb saddr LUN subpath state req/s MB/s err act act act act act act act PAS ACT PAS ACT ifcb mylpan-ps5 9/ =============================================================== TOTAL 9/ In the SCSI address column (saddr), note the device IDs of the devices configured on the pserver. Devices appear in numeric order. The top device, 0.0, corresponds to device sda on the pserver, followed by sdb, and so on. 11. In the pserver console window, enter the following: # dd if=/dev/sdc of=/dev/null 12. Examine the output that appears in the cblade console windows. Generating I/O for device sdc in this way appears as requests per second (req/s) and megabytes per second (MB/s) for SCSI address 2.0: Thu Feb 2 15:10: pblade topology pb saddr LUN subpath state req/s MB/s err act act act act act act act ACT PAS PAS PM5.2_BF

51 Beta Testing your SAN Configuration ACT ifcb mylpan-ps5 9/ ================================================================= TOTAL 9/ Thu Feb 2 15:10: Compare the numbers that appear: if the disk device on the storage array is configured on an active/active controller, within each column, the values displayed on all HBAs should be roughly equivalent, indicating that disk device s I/O is balanced properly. In this example, generated on a plaform whose cblades contain Fibre Channel HBAs, the command output shows that the pserver has a LUN that has a state of active/passive ( ). (Specifically, its active subpath is ; its passive subpath is ) To verify that this LUN is handling I/O properly, issue the following command on the pserver (using sde because it is the pserver s fifth device) to generate I/O on that device: # dd if=/dev/sde of=/dev/null Output similar to the following appears: Thu Feb 2 15:12: pblade topology pb saddr LUN subpath state req/s MB/s err act act act act act act act PAS ACT PAS ACT ifcb mylpan-ps5 9/ =============================================================== TOTAL 9/ PM5.2_BF 3-7

52 PAN Manager SAN Integration Guide Beta Notice that I/O is only occurring on HBA port 4 (the active port) of SCSI address (120.0). This means that I/O is functioning as expected. In general: You should not see I/O on any secondary paths You should not see the active/passive states change or the error count increase Each active (or primary) path on which you generate I/O should display non-zero statistics. The values for MBs/s across these active subpaths should be approximately equivalent. Certain configurations will not have the I/O load distributed equally across all active paths: On an active/active array, there should be equal I/O distribution across the LUNs subpaths On an active/passive array, or on a primary/secondary array, in which the number of active (or primary) paths does not equal the number of passive (or secondary) paths, the I/O may not be equally distributed. This is benign and can be ignored. Test I/O on Each Tape Device Note: Support for Fibre Channel tape devices requires having Fibre Channel HBAs on the platform s cblades. To ensure the flow of I/O to and from a specific SCSI tape device: 1. Insert a scratch tape in each tape device that you want to test. 2. Using a web browser, log on to the PAN Manager GUI. 3. Create an LPAN, and then create one or more pservers in the LPAN. (See PAN Manager Administrator s Guide, if necessary.). 4. Assign all SCSI tape devices equally among the pservers that you have created. 3-8 PM5.2_BF

53 Beta Testing your SAN Configuration 5. Install the same pserver software on each pserver. (This example uses the Linux operating system. Refer to the appropriate pserver Guide for instructions for installing the pserver software.) 6. Install an application or use an operating system utility that generates I/O. (In this case, we use the Linux dd command.) 7. Use the network utility of your choice (such as ssh or PuTTY) to open a remote network console on each cblade. 8. Display all SCSI tape devices assigned to a specific pserver. To do this, in the PAN Manager GUI left pane, select LPANs > lpan_name > pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. 9. Open a console on the pserver, by doing one of the following: Using a network utility of your choice (such as ssh or PuTTY), log on to one of the cblades, and then enter console lpan_name/pserver_name. In the PAN Manager GUI left pane, select LPANs > lpan_name > pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. On the pserver pserver_name page, in the bottom border, click the console icon. 10. On the pserver pserver_name page, in the SCSI Tapes area, select a tape device by clicking its SCSI ID. 11. Use this tape device s SCSI ID to display I/O statistics. (Issue the command that is appropriate for your hardware platform.) # pathstat -S u -i 1 -t This example displays I/O statistics at one-second intervals for SCSI ID , and includes the current time. 12. On the SCSI Tape tape_device_name page, in the Assignments area, note the target ID of the device on the pserver, which appears in parentheses. 13. In the pserver console window, issue the Linux dd command to generate I/O on the pserver. PM5.2_BF 3-9

54 PAN Manager SAN Integration Guide Beta Caution: The Linux tape class driver gets its maximum transfer size from the SCSI tape device. If the maximum transfer size for a tape device is greater than 128 KB, the I/O operation fails. To avoid this issue when running applications, use a transfer size of 128 KB or smaller. To do this, enter the following: # dd if=/dev/zero of=/dev/st0 bs=128k 14. Examine the output that appears in the cblade console windows: Thu Feb 2 15:50: multipath device w/ subpaths LUN subpath state req/s MB/s err act ifcb 1/ ========================================================== TOTAL 1/ Each SCSI tape device uses only a single I/O path. This command output indicates that the pserver is successfully writing data to the tape device through HBA port 3. Manage SAN I/O on the cblades In previous sections in this chapter, you verified the physical connectivity of storage resources to the PAN environment (see Validate the Physical Connectivity for SCSI Devices on page 3-2) and verified that each individual storage resource was handling I/O properly (see Test I/O on SCSI Devices on page 3-4). This section describes how to examine I/O on all configured storage devices simultaneously, by displaying statistics for an entire cblade PM5.2_BF

55 Beta Check the Load Balancing on the cblades Testing your SAN Configuration This section describes how to examine I/O statistics for the following storage environment(s): One that contains only SCSI disk resources One that contains SCSI disk and SCSI tape resources Note: Support for Fibre Channel tape devices requires having Fibre Channel HBAs on the platform s cblades. Check Disk I/O Only This section describes how to examine I/O statistics on cblades that are processing I/O for all disk devices available to the PAN. To check I/O statistics for disk devices: 1. Use the network utility of your choice (such as ssh or PuTTY) to open a remote network console on each cblade. 2. In each cblade console, run the pathtop utility: # pathtop 3. Open a console on each pserver on which SCSI disk devices are configured, by doing one of the following: Using a network utility of your choice (such as ssh or PuTTY), log on to one of the cblades, and then enter console lpan_name/pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. In the PAN Manager GUI left pane, select LPANs > lpan_name > pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. On the pserver pserver_name page, in the bottom border, click the console icon. 4. On each pserver, install an application or use an operating system utility that generates I/O using each SCSI disk device configured on it. PM5.2_BF 3-11

56 PAN Manager SAN Integration Guide Beta You can write and run a script on each pserver that performs a basic operation using the Linux dd command. For example, on a pserver that has eight SCSI disks configured, this script would generate read operations on each configured device: dd if=/dev/sda of=/dev/null & dd if=/dev/sdb of=/dev/null & dd if=/dev/sdc of=/dev/null & dd if=/dev/sdd of=/dev/null & dd if=/dev/sde of=/dev/null & dd if=/dev/sdf of=/dev/null & dd if=/dev/sdg of=/dev/null & dd if=/dev/sdh of=/dev/null & Output similar to the following appears in the cblade console windows. Each row of data pertains to a specific HBA port, numbered in the leftmost column. multipath meter Wed Jan 11 14:22: hba world-wide-name enable link que trq/s tmb/s avg-tkb avg-rkb avg-wkb e600867c use up e600867c use up Total all LUNs In one of the cblade console windows, check the following numbers: a. For each HBA port, check the value of trq/s (total requests per second). The values that appear should be roughly equal for each HBA port on the cblade. b. For each HBA port, check the value of tmb/s (total megabytes per second). The values that appear should be roughly equal for each HBA port on the cblade. 6. In the other cblade console window, check the same values that you did on the first cblade. The values that appear for trq/s (and also tmb/s) should be roughly equal for each HBA port on this cblade. 7. If you plan to test cblade I/O for SCSI disk and tape devices simultaneously (recommended if you have SAN tape devices), do the following: a. Determine on which cblade(s) you have configured SCSI tape devices PM5.2_BF

57 Beta Testing your SAN Configuration b. On each cblade on which SCSI tape devices are configured, in the row labeled Total all LUNs, note the total value that appears in the tmb/s column, in this case This value represents the total megabytes per second for all HBA ports on the cblade that are processing I/O for disk devices. You will use this value as a baseline in the next section when comparing this figure to a cblade on which SCSI tape resources are active. Check Disk I/O and Tape I/O Simultaneously Note: Support for Fibre Channel tape devices requires having Fibre Channel HBAs on the platform s cblades. This section describes how to examine I/O statistics on cblades that are processing I/O for disk and tape devices simultaneously. The purpose is to verify that you have configured your tape devices according to the best practices described in Best Practices for Configuring SCSI Tape Resources on page Caution: The Linux tape class driver gets its maximum transfer size from the SCSI tape device. If the maximum transfer size for a tape device is greater than 128 KB, the I/O operation fails. To avoid this issue when running applications, use a transfer size of 128 KB or smaller. To check I/O statistics for disk and tape devices: 1. Use the network utility of your choice (such as ssh or putty) to open a remote network console on each cblade. 2. In each cblade console, run the pathtop utility: # pathtop 3. Open a console on each pserver on which SCSI disk and SCSI tape devices are configured, by doing one of the following: PM5.2_BF 3-13

58 PAN Manager SAN Integration Guide Beta Using a network utility of your choice (such as ssh or putty), log on to one of the cblades, and then enter console lpan_name/pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. In the PAN Manager GUI left pane, select LPANs > lpan_name > pserver_name, where lpan_name and pserver_name indicate the name of your LPAN and pserver, respectively. On the pserver pserver_name page, in the bottom border, click the console icon. 4. On each pserver, install an application or use an operating system utility that generates I/O using each SCSI disk and tape device configured on it. You can write and run a script on each pserver that performs a basic operation using the Linux dd command. For example, on a pserver that has eight SCSI disks and one SCSI tape device configured, this script would generate read operations on each configured device: dd if=/dev/sda of=/dev/null & dd if=/dev/sdb of=/dev/null & dd if=/dev/sdc of=/dev/null & dd if=/dev/sdd of=/dev/null & dd if=/dev/sde of=/dev/null & dd if=/dev/sdf of=/dev/null & dd if=/dev/sdg of=/dev/null & dd if=/dev/sdh of=/dev/null & dd if=/dev/zero of=/dev/st0 bs=128k & (This sample script is identical to the one we suggest in the previous section, except that it has an extra line, for a tape device.) Output similar to the following appears in the cblade console windows. Each row of data pertains to a specific HBA port, numbered in the leftmost column. multipath meter Wed Jan 11 14:27: hba world-wide-name enable link que trq/s tmb/s avg-tkb avg-rkb avg-wkb e600867b use up e600867b use up Total all LUNs PM5.2_BF

59 Beta Testing your SAN Configuration 5. In one of the cblade console windows, check the following numbers: a. For each HBA port, check the value of trq/s (total requests per second). In this example, note that the value for HBA4 is significantly higher than that for HBA3. This indicates that SCSI tape devices are attached to HBA3. b. For each HBA port, check the value of tmb/s (total megabytes per second). Note again that the value for HBA4 is significantly higher than that for HBA3, indicating that SCSI tape devices are attached to HBA3. 6. In the other cblade console window, check the same values that you did on the first cblade. If you have SCSI tape devices configured on this cblade, the values for trq/s and tmb/s will be lower for the HBA(s) that have SCSI tape devices configured on them. 7. On each cblade on which SCSI tape devices are configured, in the row labeled Total all LUNs, note the total value for all HBA ports that appears in the tmb/s column. 8. Identify which HBA port has tape devices configured on it, note its value for tmb/s, and then subtract this value from the total tmb/s value for all HBAs. This value represents the total megabytes per second for all HBA ports on the cblade that are processing I/O for disk devices. 9. Compare the value you derived in Step 8 to the similar value you displayed at the end of the previous section ( Check Disk I/O Only on page 3-11). If the two values vary significantly, do the following, in order: a. Be sure that you have followed the guidelines for configuring SCSI tape devices in Best Practices for Configuring SCSI Tape Resources on page b. Contact Fujitsu Technology Solutions customer support or your authorized support vendor for assistance. PM5.2_BF 3-15

60 PAN Manager SAN Integration Guide Beta Check I/O Statistics on Both cblades This procedure describes how to check overall I/O statistics for an entire cblade, and to compare this value to the same value on the other cblade. The goal is to have the I/O load distributed evenly between the two cblades. If you plan to configure SCSI tape devices, we recommend that you do the following: Perform this procedure first without any tape devices running. This allows you to determine whether disk I/O is properly balanced across the cblades. Perform this procedure a second time with SCSI tape devices running. In order for this test to be as accurate as possible, configure your pservers (tape media servers) exactly as you would in a production environment. For additional guidelines for configuring tape devices, see Best Practices for Configuring SCSI Tape Resources on page To check overall I/O statistics, run the pathtop utility in each cblade console: # pathtop This generates output similar to the following: multipath meter Fri Feb 3 13:40: hba world-wide-name enable link que trq/s tmb/s avg-tkb avg-rkb avg-wkb e600867c use up e600867c use up Total all LUNs hba io requests/sec <= size (bytes) scsi type any 512 1k 2k 4k 8k 16k 32k 64k 128k 3 read write total read write total The total number of megabytes per second (tmb/s), here , should be roughly equivalent for cblade A and cblade B PM5.2_BF

61 Beta Testing your SAN Configuration Take a Snapshot of the Finished SAN Configuration Note: You can take a snapshot of your SAN configuration in this way only if your SAN uses the active/passive pathing model. After you are satisfied that the SAN is configured properly, and functioning properly, we recommend that you capture its finished configuration by running the pathctl utility. This can be helpful later, especially when you want to correct I/O imbalances that result from a SAN failover. To preserve a snapshot of the pathing models that are configured on the storage array: 1. Open a console window on a cblade. 2. Run the pathctl utility: # pathctl list more Output similar to the following appears: Wed Jan 11 15:03: Active/Passive luns: Path Group 0 Path 15 ( 9, 0,2,0) serial APM wwn: d0000ba4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,0) ACTIVE Device 1:( 4, 0,2,0) PASSIVE Path 16 ( 9, 0,2,1) serial APM wwn: d0000bb4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,1) PASSIVE Device 1:( 4, 0,2,1) ACTIVE Path 17 ( 9, 0,2,2) serial APM wwn: d0000bc4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,2) ACTIVE Device 1:( 4, 0,2,2) PASSIVE Path 18 ( 9, 0,2,3) serial APM wwn: d0000bd4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,3) PASSIVE Device 1:( 4, 0,2,3) ACTIVE... PM5.2_BF 3-17

62 PAN Manager SAN Integration Guide Beta (This output is typical of an active/passive pathing model.) Keep this output to use later, when comparing the path model for an array that has been running. Detect and Correct I/O Imbalances Although SAN failover allows I/O to continue, a SAN configuration in which failover has occurred may contain uneven concentrations of active LUNs. (The SAN combines LUNs originally configured as active, and LUNs that were passive but are now active.) The following procedures: Help detect the active/passive status of LUNs in a SAN in which a failover event has taken place Describe how to trespass specific LUNs (that is, change their active/passive status) Detect an I/O Imbalance To detect a difference in the configuration of LUNs between the SAN s current configuration and its original configuration: 1. Open a console window on a cblade. 2. Run the pathctl utility: # pathctl list more 3. Compare this output to the output you captured before the SAN failover occurred (see Take a Snapshot of the Finished SAN Configuration on page 3-17). Wed Feb 18 14:13: Active/Passive luns: Path Group 0 Path 15 ( 9, 0,2,0) serial APM wwn: d0000ba4bdf9d331cd811 state:normal Clariion 3-18 PM5.2_BF

63 Beta Testing your SAN Configuration Device 0:( 3, 0,2,0) ACTIVE Device 1:( 4, 0,2,0) PASSIVE Path 16 ( 9, 0,2,1) serial APM wwn: d0000bb4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,1) ACTIVE Device 1:( 4, 0,2,1) PASSIVE Path 17 ( 9, 0,2,2) serial APM wwn: d0000bc4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,2) ACTIVE Device 1:( 4, 0,2,2) PASSIVE Path 18 ( 9, 0,2,3) serial APM wwn: d0000bd4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,3) ACTIVE Device 1:( 4, 0,2,3) PASSIVE... Reading from top to bottom, each device s four values within parentheses indicate HBA port number, path group, target, and LUN, respectively. In this output, each path shows that all I/O is passing through HBA port 3, and none through HBA port 4. (Contrast this to the alternating active and passive status for device 0 in the example in Take a Snapshot of the Finished SAN Configuration on page 3-17.) Correct an I/O Imbalance If, after examining the output from the pathctl utility, you determine that there are concentrations of active LUNs, the cause might be one of the following: Some paths might be broken. If the HBA port numbers that appear are different from those in your original configuration, check the HBA connections on the cblades, and then rerun the pathctl utility. A SAN failover event may have caused some passive paths to become active, and/or some active paths to become passive. A storage processor in the storage array may have rebooted or is down, causing LUNs to be trespassed to the remaining storage processor. (Trespassing a LUN means activating a passive path when I/O has failed on the corresponding active path.) PM5.2_BF 3-19

64 PAN Manager SAN Integration Guide Beta To restore the active/passive status of all LUNs to their original state: 1. Determine which LUNs you want to trespass. 2. Open a console window on a cblade. 3. Run the pathctl utility: # pathctl trespass group x path y device z where: x indicates the number of the path group. (If you do not have path groups configured, this number is zero.) y indicates the path number. z indicates a single device number within a specific path. For example, to restore two paths to their previous status, such as Path 16 and Path 18, enter the following: # pathctl trespass group 0 path 16 device 1 # pathctl trespass group 0 path 18 device 1 4. To verify that these trespass operations were successful, issue the pathctl utility again: # pathctl list more Note the differences compared to the output you generated in the previous section. Wed Feb 18 14:15: Active/Passive luns: Path Group 0 Path 15 ( 9, 0,2,0) serial APM wwn: d0000ba4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,0) ACTIVE Device 1:( 4, 0,2,0) PASSIVE Path 16 ( 9, 0,2,1) serial APM wwn: d0000bb4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,1) PASSIVE Device 1:( 4, 0,2,1) ACTIVE Path 17 ( 9, 0,2,2) serial APM wwn: d0000bc4bdf9d331cd811 state:normal Clariion Device 0:( 3, 0,2,2) ACTIVE Device 1:( 4, 0,2,2) PASSIVE Path 18 ( 9, 0,2,3) serial APM wwn: d0000bd4bdf9d331cd811 state:normal Clariion 3-20 PM5.2_BF

65 Beta Testing your SAN Configuration... Device 0:( 3, 0,2,3) PASSIVE Device 1:( 4, 0,2,3) ACTIVE If the results of the trespass operations are not what you expected, contact Fujitsu Technology Solutions customer support or your authorized support vendor for assistance. Detecting a Possible Zoning Issue This procedure describes how to use the pathstat topology report (-T) to detect a load imbalance for a device on a cblade. To display the topology report: 1. Use the Linux dd command to generate SCSI device I/O on a pserver. 2. Run this command: # pathstat -T 1 -u -t -i 1 Compare the output on cblade A to the output on cblade B. Output similar to the following appears on cblade A: Thu Feb 2 16:13: pblade topology pb saddr LUN subpath state req/s MB/s err act act act act act act act ACT PAS PAS ACT ifca mylpan-ps5 9/ =============================================================== TOTAL 9/ Output similar to the following appears on cblade B: PM5.2_BF 3-21

66 PAN Manager SAN Integration Guide Beta Thu Feb 2 16:14: pblade topology pb saddr LUN subpath state req/s MB/s err act act act act act act act act ACT PAS PAS ACT act ifcb mylpan-ps5 11/ =============================================================== TOTAL 11/ Notice that on cblade A, SCSI address (saddr) 0.0 has only one path; on cblade B the same SCSI address has two paths. This could indicate a zoning issue that prevents subpath from appearing on cblade A. If this occurs: a. Check the Fibre Channel cables leading from the fibre switch (if any) to the storage array. b. Verify that zoning is properly configured on your fabric switch. c. Note that because there is a tape device configured on HBA port 3 of cblade B (SCSI address 224.0), it is processing all I/O through HBA port 3, at the expense of other active/ active devices whose subpaths use this port (those numbered 3.n.n.n). It is therefore acceptable for there to be zeroes in the req/s and MB/s columns for devices that use these subpaths. d. If the problem persists, contact Fujitsu Technology Solutions customer support or your authorized support vendor PM5.2_BF

67 Beta Testing your SAN Configuration PM5.2_BF 3-23

68 PAN Manager SAN Integration Guide Beta 3-24 PM5.2_BF

69 Appendixes This section of PAN Manager SAN Integration Guide contains the following appendixes: Appendix A, Troubleshooting and Maintenance Appendix B, Using SAN Diagnostic Tools Appendix C, SAN Resource Guidelines and Limitations

70 PAN Manager SAN Integration Guide

71 Beta Appendix A Troubleshooting and Maintenance This appendix lists error situations that might occur during the SAN integration process, and provides some troubleshooting hints and recovery and maintenance suggestions. This appendix includes the following sections: SAN Configuration Issues Troubleshooting Recoverability Maintenance Mode Diagnosing and Correcting Performance Issues SAN Failover The BladeFrame BF400 S2 and BladeFrame BF200 hardware platform integrates with a wide range of SAN storage systems and have been qualified on many hardware types and configurations. Contact Fujitsu Technology Solutions customer support or your authorized support vendor to verify that your storage hardware has been tested and qualified. PM5.2_BF A-1

72 PAN Manager SAN Integration Guide SAN Configuration Issues Beta If you have questions about the software or firmware versions your storage array requires, contact Fujitsu Technology Solutions customer support or your authorized support vendor. Troubleshooting The following table provides a list of issues that you might encounter and suggested steps to troubleshoot and resolve each issue. Table A.1 Troubleshooting SAN Issues Issue Observed Multipath drivers are disabled Troubleshooting Hints The multipath drivers are enabled by default and cannot be disabled. If you have disabled the generic multipath driver, contact Fujitsu Technology Solutions customer support or your authorized support vendor for instructions specific to your SAN hardware configuration. A-2 PM5.2_BF

73 Beta Troubleshooting and Maintenance Issue Observed After running the pathtop utility, the utility s output displays the value Down in the link column. Troubleshooting Hints There is no link light on the HBA adapter, and could indicate one of the following: Faulty HBA Bad SAN cable Bad GBIC on the switch port Faulty switch Run the pathtop utility. In the command output, each row consists of data for a specific HBA port. The value in the Link column indicates whether the HBA port is 'up or DOWN. (See Check I/O Statistics on Both cblades on page 3-16 for information about using the pathtop utility.) Recoverability You must recover any storage processor failures manually. The following failures and errors should recover automatically, except where noted: Link failures Switch failures Zoning and LUN masking errors should recover automatically during system operation, but you must recover them manually at startup. HBA failures should recover automatically, but you must recover them manually if there are LUN mismatches. PM5.2_BF A-3

74 PAN Manager SAN Integration Guide Maintenance Mode Beta Your cblades are equipped with a utility called CIO (Customer Information Operation). CIO will automatically alert Fujitsu Technology Solutions customer support or your authorized support vendor in the case of a severe cblade event, such as an unresponsive cblade. However, there may be times when such an event is expected, typically during routine maintenance. In such cases, use the maintenance mode feature to prevent an alert being sent to service personnel. To turn on maintenance mode: 1. Log on to a cblade. 2. Change to the /opt/panmgr/bin directory. 3. At the prompt, enter: # maintenance on To turn off maintenance mode: 1. Log on to a cblade. 2. Change to the /opt/panmgr/bin directory. 3. At the prompt, enter: # maintenance off To check the status of maintenance mode: 1. Log on to a cblade. 2. Change to the /opt/panmgr/bin directory. 3. At the prompt, enter: # maintenance status A message appears stating whether maintenance mode is on or off. A-4 PM5.2_BF

75 Beta Troubleshooting and Maintenance Diagnosing and Correcting Performance Issues Tasks common to the maintenance of an integrated SAN and platform include running reports and monitoring system logs to look for anomalies and potential SAN problems, such as failed ports or I/O bottlenecks. This section describes the cblade software tools and utilities you can use to diagnose and resolve performance issues related to the internal network and I/O. The cblade software provides the utilities listed in this section to help monitor and reconfigure I/O paths. For information about using diagnostic tools to test your SAN configuration, see Chapter 3, Testing your SAN Configuration. Diagnostic Tools There are a number of diagnostic utilities in the cblade software, as well as standard Linux and Windows diagnostic tools, that you can use to monitor traffic between a platform and a SAN. Diagnostic Utilities in the Control Blade Software The cblade software contains the following diagnostic tools that can help you detect and correct integration issues with your SAN subsystem: giscsimon Reads the multipath statistics and debugging files, and reports information captured by the giscsimon daemon. pathctl Displays the current multipath I/O configuration, and for platforms in which the cblades contain Fibre Channel HBAs, allows you to reconfigure (or trespass) LUNs on active/ passive storage arrays. pathstat Provides device performance monitoring statistics for SAN I/O subpaths, SCSI disks, and SCSI tapes (depending on your storage array type and pserver type). PM5.2_BF A-5

76 PAN Manager SAN Integration Guide Beta pathtop Reports performance statistics related to specific HBAs on the cblade. Examples for using pathctl, pathstat, and pathtop utilities to detect and resolve SAN configuration issues appear in Chapter 3, Testing your SAN Configuration. For more information about these utilities, see Appendix B, Using SAN Diagnostic Tools. Linux and Windows Diagnostic Tools Linux tools can provide a window into SAN network traffic to identify and isolate I/O bottlenecks and failed ports. Use the iostat and vmstat commands to report central processing statistics. The top command also provides a snapshot of processor activity in realtime and displays a listing of the most CPU-intensive tasks on the system. Refer to the Linux man pages for additional information about these commands. Windows Server systems also have a strong set of performance monitoring tools (system monitor, performance logs, and alerts) that you can use to diagnose systems issues specific to pservers that are running Windows Server. See the appropriate pserver Guide for Microsoft Windows Server for details. SAN Failover This section describes the conditions that can disrupt all SAN connectivity, and what happens during connection reinitialization and connection failback. Specifically, the following sections provide the background and guidelines for handling SAN failover: Conditions That Disrupt All SAN Connectivity Connection Reinitialization A-6 PM5.2_BF

77 Beta Troubleshooting and Maintenance Connection Failback Conditions That Disrupt All SAN Connectivity When properly configured, the hardware platform provides redundant I/O connectivity to a SAN. However, there are specific, unique conditions that disrupt all SAN connectivity. Figure A.1 Possible SAN Disruption Points Figure A.1 illustrates points in the network where SAN disruptions can occur. The following scenarios can cause SAN disruptions: Removing and reconnecting one Fibre Channel cable on a cblade and then removing the second connection in the pair before the first connection reinitializes causes a disruption on all pserver I/O requests through that connection pair. Removing only the send or receive portion of the Fibre Channel connection causes the entire connection to fail, and disrupts I/O. If the connection is not restored, pending I/O requests fail over to an active connection. PM5.2_BF A-7