Hardware and Service Guide

Transcription

1 IBM System Storage EXN3000 Storage Expansion Unit Hardware and Serice Guide GC

2 Note: Before using this information and the product it supports, be sure to read the general information in Notices on page 63. The following paragraph does not apply to any country (or region) where such proisions are inconsistent with local law. INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states (or regions) do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you. Order publications through your IBM representatie or the IBM branch office sering your locality. Copyright IBM Corporation 2009, US Goernment Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

3 Safety notices Safety notices and labels This section contains information about: Safety notices and labels Laser safety on page i Rack safety on page iii Handling fiber-optic cables on page x Fire suppression systems on page xi For information about enironmental notices, see the IBM Enironmental Notices and User Guide. When using this product, obsere the danger, caution, and attention notices contained in this guide. The notices are accompanied by symbols that represent the seerity of the safety condition. The following sections define each type of safety notice and proide examples. The following notices and statements are used in IBM documents. They are listed below in order of increasing seerity of potential hazards. Follow the links for more detailed descriptions and examples of the danger, caution, and attention notices in the sections that follow. Note: These notices proide important tips, guidance, or adice. Attention notices on page i: These notices indicate potential damage to programs, deices, or data. Caution notices on page i: These statements indicate situations that can be potentially hazardous to you. Danger notices : These statements indicate situations that can be potentially lethal or extremely hazardous to you. Safety labels are also attached directly to products to warn of these situations. In addition to these notices, Labels on page may be attached to the product to warn of potential hazards. Danger notices About this task A danger notice calls attention to a situation that is potentially lethal or extremely hazardous to people. A lightning bolt symbol accompanies a danger notice to represent a dangerous electrical condition. A sample danger notice follows. Copyright IBM Corp. 2009, 2014 iii

4 DANGER An electrical outlet that is not correctly wired could place hazardous oltage on metal parts of the system or the deices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to preent an electrical shock. (D004) A general danger notice proides instructions on how to aoid shock hazards when sericing equipment. Unless instructed otherwise, follow the procedures in the following danger notice. i IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

5 DANGER When working on or around the system, obsere the following precautions: Electrical oltage and current from power, telephone, and communication cables are hazardous. To aoid a shock hazard: Connect power to this unit only with the IBM proided power cord. Do not use the IBM proided power cord for any other product. Do not open or serice any power supply assembly. Do not connect or disconnect any cables or perform installation, maintenance, or reconfiguration of this product during an electrical storm. The product might be equipped with multiple power cords. To remoe all hazardous oltages, disconnect all power cords. Connect all power cords to a properly wired and grounded electrical outlet. Ensure outlet supplies proper oltage and phase rotation according to the system rating plate. Connect any equipment that will be attached to this product to properly wired outlets. When possible, use one hand only to connect or disconnect signal cables. Neer turn on any equipment when there is eidence of fire, water, or structural damage. Disconnect the attached power cords, telecommunications systems, networks, and modems before you open the deice coers, unless instructed otherwise in the installation and configuration procedures. Connect and disconnect cables as described below when installing, moing, or opening coers on this product or attached deices. To Disconnect: 1. Turn off eerything (unless instructed otherwise). 2. Remoe power cords from the outlet. 3. Remoe signal cables from connectors. 4. Remoe all cables from deices. To Connect: 1. Turn off eerything (unless instructed otherwise). 2. Attach all cables to deices. 3. Attach signal cables to the connectors. 4. Attach power cords to the outlets. 5. Turn on the deices. (D005) Labels As an added precaution, safety labels are often installed directly on products or product components to warn of potential hazards. Safety notices

6 The actual product safety labels may differ from these sample safety labels: DANGER Hazardous oltage, current, or energy leels are present inside any component that has this label attached. Do not open any coer or barrier that contains this label. (L001) (L003) DANGER Multiple power cords. The product might be equipped with multiple power cords. To remoe all hazardous oltages, disconnect all power cords. Caution notices A caution notice calls attention to a situation that is potentially hazardous to people because of some existing condition. A caution notice can be accompanied by different symbols, as in the examples below: If the symbol is... It means... A hazardous electrical condition with less seerity than electrical danger. A generally hazardous condition not represented by other safety symbols. A hazardous condition due to the use of a laser in the product. Laser symbols are always accompanied by the classification of the laser as defined by the U. S. Department of Health and Human Serices (for example, Class I, Class II, and so forth). Attention notices An attention notice indicates the possibility of damage to a program, deice, or system, or to data. An exclamation point symbol may accompany an attention notice, but is not required. A sample attention notice follows: Attention: Do not bend a fiber cable to a radius less than 5 cm (2 in.); you can damage the cable. Tie wraps are not recommended for optical cables because they can be easily oertightened, causing damage to the cable. Laser safety CAUTION: This product contains a Class 1M laser. Do not iew directly with optical instruments. (C028) i IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

7 This equipment contains Class 1 laser products, and complies with FDA radiation Performance Standards, 21 CFR Subchapter J and the international laser safety standard IEC CAUTION: Data processing enironments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power leels. For this reason, neer look into the end of an optical fiber cable or open receptacle. (C027) Attention: In the United States, use only SFP or GBIC optical transceiers that comply with the FDA radiation performance standards, 21 CFR Subchapter J. Internationally, use only SFP or GBIC optical transceiers that comply with IEC standard Optical products that do not comply with these standards may produce light that is hazardous to the eyes. Usage restrictions The optical ports of the modules must be terminated with an optical connector or with a dust plug. Safety notices ii

8 Rack safety Rack installation DANGER Obsere the following precautions when working on or around your IT rack system: Heay equipment - personal injury or equipment damage might result if mishandled. Always lower the leeling pads on the rack cabinet. Always install stabilizer brackets on the rack cabinet. To aoid hazardous conditions due to uneen mechanical loading, always install the heaiest deices in the bottom of the rack cabinet. Always install serers and optional deices starting from the bottom of the rack cabinet. Rack-mounted deices are not to be used as sheles or work spaces. Do not place objects on top of rack-mounted deices. Each rack cabinet might hae more than one power cord. Be sure to disconnect all power cords in the rack cabinet when directed to disconnect power during sericing. Connect all deices installed in a rack cabinet to power deices installed in the same rack cabinet. Do not plug a power cord from a deice installed in one rack cabinet into a power deice installed in a different rack cabinet. An electrical outlet that is not correctly wired could place hazardous oltage on the metal parts of the system or the deices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to preent an electrical shock. (R001 part 1 of 2) iii IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

9 CAUTION: Do not install a unit in a rack where the internal rack ambient temperatures will exceed the manufacturer's recommended ambient temperature for all your rack-mounted deices. Do not install a unit in a rack where the air flow is compromised. Ensure that air flow is not blocked or reduced on any side, front, or back of a unit used for air flow through the unit. Consideration should be gien to the connection of the equipment to the supply circuit so that oerloading of the circuits does not compromise the supply wiring or oercurrent protection. To proide the correct power connection to a rack, refer to the rating labels located on the equipment in the rack to determine the total power requirement of the supply circuit. (For sliding drawers.) Do not pull out or install any drawer or feature if the rack stabilizer brackets are not attached to the rack. Do not pull out more than one drawer at a time. The rack might become unstable if you pull out more than one drawer at a time. (For fixed drawers) This drawer is a fixed drawer and should not be moed for sericing unless specified by manufacturer. Attempting to moe the drawer partially or completely out of the rack may cause the rack to become unstable or cause the drawer to fall out of the rack. (R001 part 2 of 2) Safety notices ix

10 Rack relocation (19" rack) Handling fiber-optic cables CAUTION: Remoing components from the upper positions in the rack cabinet improes rack stability during relocation. Follow these general guidelines wheneer you relocate a populated rack cabinet within a room or building: Reduce the weight of the rack cabinet by remoing equipment starting at the top of the rack cabinet. When possible, restore the rack cabinet to the configuration of the rack cabinet as you receied it. If this configuration is not known, you must do the following: Remoe all deices in the 32U position and aboe. Ensure that the heaiest deices are installed in the bottom of the rack cabinet. Ensure that there are no empty U-leels between deices installed in the rack cabinet below the 32U leel. If the rack cabinet you are relocating is part of a suite of rack cabinets, detach the rack cabinet from the suite. Inspect the route that you plan to take when moing the rack to eliminate potential hazards. Verify that the route that you choose can support the weight of the loaded rack cabinet. Refer to the documentation that came with your rack cabinet for the weight of a loaded rack cabinet. Verify that all door openings are at least 760 x 2030 mm (30 x 80 in.). Ensure that all deices, sheles, drawers, doors, and cables are secure. Ensure that the four leeling pads are raised to their highest position. Ensure that there is no stabilizer bracket installed on the rack cabinet during moement. Do not use a ramp inclined at more than ten degrees. Once the rack cabinet is in the new location, do the following: - Lower the four leeling pads. - Install stabilizer brackets on the rack cabinet. - If you remoed any deices from the rack cabinet, repopulate the rack cabinet from the lowest position to the highest position. If a long distance relocation is required, restore the rack cabinet to the configuration of the rack cabinet as you receied it. Pack the rack cabinet in the original packaging material, or equialent. Also, lower the leeling pads to raise the casters off of the pallet and bolt the rack cabinet to the pallet. (R002) Before you use fiber-optic cables, read the following precautions. Attention: To aoid damage to the fiber-optic cables, follow these guidelines: Do not route the cable along a folding cable-management arm. When attaching to a deice on slide rails, leae enough slack in the cable so that it does not bend to a radius of less than 38 mm (1.5 in.) when extended or become pinched when retracted. x IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

11 Route the cable away from places where it can be snagged by other deices in the rack cabinet. Do not oertighten the cable straps or bend the cables to a radius of less than 38 mm (1.5 in.). Do not put excess weight on the cable at the connection point. Be sure that the cable is well supported. CAUTION: Data processing enironments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power leels. For this reason, neer look into the end of an optical fiber cable or open receptacle. Fire suppression systems A fire suppression system is the responsibility of the customer. The customer's own insurance underwriter, local fire marshal, or a local building inspector, or both, should be consulted in selecting a fire suppression system that proides the correct leel of coerage and protection. IBM designs and manufactures equipment to internal and external standards that require certain enironments for reliable operation. Because IBM does not test any equipment for compatibility with fire suppression systems, IBM does not make compatibility claims of any kind nor does IBM proide recommendations on fire suppression systems. Safety notices xi

12 xii IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

13 Contents Safety notices iii Safety notices and labels iii Danger notices iii Labels Caution notices i Attention notices i Laser safety i Usage restrictions ii Rack safety iii Rack installation iii Rack relocation (19" rack) x Handling fiber-optic cables x Fire suppression systems xi Figures xii Tables xix About this document xxi Audience xxi Supported features xxi Websites xxi Getting information, help, and serice xxii Before you call xxii Using the documentation xxii Hardware serice and support xxiii Firmware updates xxiii Terminology and conentions used in this document xxiii Actie/actie and high-aailability terms xxi Storage terms xxi Command conentions xx Formatting conentions xx Keyboard conentions xx How to send your comments xxi Introduction to the EXN3000 storage expansion unit EXN3000 storage expansion unit hardware oeriew Terms and symbols Important differences between EXN3000 and EXN1000/2000/4000 storage expansion units Mechanical differences Infrastructure differences Resiliency differences Information you need to understand about EXN3000 storage expansion units Minimum software requirements SAS and SATA disk drie support Power supplies Disk bay numbering Storage expansion unit IDs Unique ID per system: The 30-second rule: Configured systems: Add-on storage expansion units: Duplicate storage expansion unit IDs: Disk dries Copyright IBM Corp. 2009, 2014 xiii

14 Supported disk dries Disk drie addressing FC/SAS bridge for stretch or fabric-attached MetroCluster systems Supporting FC/SAS bridge configurations Hardware specifications Installing a storage expansion unit for a new storage system Installation safety rules Power and electrical Handling static-sensitie deices Installation requirements Required manuals, tools, and equipment Required tools and equipment for installation Checking shipment package contents Roadmap for new storage system installations Installing a storage expansion unit for a new storage system installation Changing the storage expansion unit ID Hot-adding a storage expansion unit to an existing system Requirements for hot-adding storage expansion units Error messages Hot-adding tasks Installing a storage expansion unit for a hot-add Cabling a storage expansion unit Hot-adding a storage expansion unit to an existing stack of storage expansion units Hot-adding a storage expansion unit to an existing SAS HBA or onboard SAS port Cabling a storage expansion unit to an existing SAS HBA or onboard SAS port Verifying the disk drie, storage expansion unit, and ACP firmware ersions Managing the storage expansion unit Storage expansion unit management safety rules Monitoring storage expansion unit components Monitoring the front operator display panel Operator display panel console error messages Monitoring storage expansion unit components Monitoring the disk drie Disk drie console error messages Monitoring the IOMs IOM console error messages Monitoring the power supply Power supply console error messages Replacing storage expansion unit components Replacing SAS cables Replacing SAS cables in a multipath HA or single-controller dual-path (multipath) configuration Replacing SAS cables in a single-controller-single-path or single-controller-mixed-path configuration Hot-swapping a disk drie Remoing a disk drie Installing a disk drie Hot-swapping or replacing an IOM About this procedure Hot-swapping or replacing an IOM Replacing a power supply Remoing a power supply Installing a power supply Recommended Power Line Sizes Recommended AC power line sizes FRU/CRU and power cord list for N series products FRU/CRU list for N series products xi IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

15 Power cord list for N series products Notices Trademarks Important notes Electronic emission notices Federal Communications Commission Statement Industry Canada Compliance Statement Australia and New Zealand Class A Statement European Union Electromagnetic Compatibility Directie Germany Electromagnetic Compatibility Directie People's Republic of China Class A Statement Taiwan Class A Statement Taiwan Contact Information Japan Voluntary Control Council for Interference Class A Statement Japan Electronics and Information Technology Industries Association Statement Korean Communications Commission Class A Statement Russia Electromagnetic Interference Class A Statement Power cords Index Contents x

16 xi IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

17 Figures 1. Front iew of the EXN3000 storage expansion unit Rear iew of the EXN3000 storage expansion unit with two power supplies Rear iew of the EXN3000 storage expansion unit with four power supplies Operator display panel LEDs Disk drie LED indicators IOM LED indicators Power supply unit LEDs Releasing a disk drie Remoing a disk drie Releasing an IOM Remoing an IOM Releasing a power supply Remoing a power supply Copyright IBM Corp. 2009, 2014 xii

18 xiii IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

19 Tables 1. Mechanical differences between SAS and FC storage expansion units Infrastructure differences between SAS and FC storage expansion units Resiliency differences between SAS and FC storage expansion units Disk drie addressing EXN3000 physical characteristics EXN3000 clearance dimensions EXN3000 enironmental requirements EXN3000 electrical requirements EXN3000 electrical requirements SAS dries EXN3000 electrical requirements 4TB encrypted and unencrypted SAS disk dries: EXN3000 electrical requirements SATA dries Electrical requirements SATA dries, GB or GB disk dries: Electrical requirements SATA dries, 6-100GB SSD dries with 18-1 TB or 18-3 TB disk dries: Hot-adding error messages Operator display panel LEDs Operator display panel console error messages Disk drie LEDs Disk drie console error messages IOM LEDs IOM console error messages Power supply LEDs Power supply console error messages Recommended conductor size for 2% oltage drop American Wire Gauge (AWG) to Harmonized Cordage Copyright IBM Corp. 2009, 2014 xix

20 xx IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

21 About this document Audience Supported features Websites This guide proides information about the IBM System Storage EXN3000 storage expansion unit (model number ), including installing the storage expansion unit, monitoring LEDs, and replacing storage expansion unit components. Compliance ID 2857-NAS coers the model For the latest ersion of this document and all IBM System Storage N series documentation, go to the IBM N series support website, which is accessed and naigated as described in Websites. For a list of N series hardware and hardware-related documents, as well as lists of Data ONTAP documentation by release family, refer to the "Bibliography" appendix in the IBM System Storage N series Introduction and Planning Guide. This guide addresses setup, operation, and sericing of the EXN3000 storage expansion unit (model number ). This document is intended to proide information to customers, operators, administrators, installers, and serice personnel who are familiar with operating systems that run on the storage system's clients, such as UNIX, Windows, and Microsoft Windows Serer. The procedures in this guide describe replacement, upgrade, and maintenance tasks for personnel with the following skills and experience: Working familiarity with small computer system hardware and operation Basic understanding of common networking concepts and practices Working familiarity with accepted tools and procedures for installing and operating sensitie electronic equipment IBM System Storage N series storage systems are drien by NetApp Data ONTAP software. Some features described in the product software documentation are neither offered nor supported by IBM. Please contact your local IBM representatie or reseller for further details. Information about supported features can also be found on the N series support website, which is accessed and naigated as described in Websites. IBM maintains pages on the World Wide Web where you can get the latest technical information and download deice driers and updates. The following web pages proide N series information: A listing of currently aailable N series products and features can be found at the following web page: Copyright IBM Corp. 2009, 2014 xxi

22 The IBM System Storage N series support website requires users to register in order to obtain access to N series support content on the web. To understand how the N series support web content is organized and naigated, and to access the N series support website, refer to the following publicly accessible web page: This web page also proides links to AutoSupport information as well as other important N series product resources. IBM System Storage N series products attach to a ariety of serers and operating systems. To determine the latest supported attachments, go to the IBM N series interoperability matrix at the following web page: For the latest N series hardware product documentation, including planning, installation and setup, and hardware monitoring, serice and diagnostics, see the IBM N series Information Center at the following web page: publib.boulder.ibm.com/infocenter/nasinfo/nseries/index.jsp Getting information, help, and serice If you need help, serice, or technical assistance or just want more information about IBM products, you will find a wide ariety of sources aailable from IBM to assist you. This section contains information about where to go for additional information about IBM and IBM products, what to do if you experience a problem with your IBM N series product, and whom to call for serice, if it is necessary. The following applies in Taiwan: IBM Taiwan Product Serice Contact Info: IBM Taiwan Corporation 3F, No 7, Song Ren Rd., Taipei Taiwan Tel: Before you call Before you call, make sure you hae taken these steps to try to sole the problem yourself: Check all cables to make sure they are connected. Check the power switches to make sure the system is turned on. Use the troubleshooting information in your system documentation and use the diagnostic tools that come with your system. Refer to the IBM support website for information on known problems and limitations. Using the documentation The latest ersions of N series software documentation, including Data ONTAP and other software products, are aailable on the IBM N series support website, which is accessed and naigated as described in Websites on page xxi. xxii IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

23 Current N series hardware product documentation is shipped with your hardware product in printed documents or as PDF files on a documentation CD. For the latest N series hardware product documentation PDFs, go to the IBM N series support website. Hardware documentation, including planning, installation and setup, and hardware monitoring, serice, and diagnostics, is also proided in an IBM N series Information Center at the following web page: Hardware serice and support You can receie hardware serice through IBM Integrated Technology Serices. Visit the following web page for support telephone numbers: Firmware updates IBM N series product firmware is embedded in Data ONTAP. As with all deices, it is recommended that you run the latest leel of firmware. Any firmware updates are posted to the IBM N series support website, which is accessed and naigated as described in Websites on page xxi. Note: If you do not see new firmware updates on the IBM N series support website, you are running the latest leel of firmware. Verify that the latest leel of firmware is installed on your machine before contacting IBM for technical support. Terminology and conentions used in this document This guide uses the following terminology, command conentions, format conentions and keyboard conentions. In this document, the term gateway describes IBM N series storage systems that hae been ordered with gateway functionality. Gateways support arious types of storage, and they are used with third-party disk storage systems. In this case, disk storage for customer data and the RAID controller functionality is proided by the back-end disk storage system. A gateway might also be used with disk storage expansion units specifically designed for the IBM N series models. The term filer describes IBM N series storage systems that either contain internal disk storage or attach to disk storage expansion units specifically designed for the IBM N series storage systems. Filer storage systems do not support using third-party disk storage systems. For information about the supported disk storage expansion units for your N series filer, refer to the IBM System Storage N series Introduction and Planning Guide. This guide, as well as other N series documentation, is aailable on the IBM N series support website, which is accessed and naigated as described in Websites on page xxi. The terms system or storage system refer to either a gateway by itself or a filer, either by itself or with additional disk dries. About this document xxiii

24 Actie/actie and high-aailability terms actie/actie configuration In the Data ONTAP 7.2 and 7.3 release families, refers to a pair of storage systems (sometimes called nodes) configured to sere data for each other if one of the two systems stops functioning. Also sometimes referred to as actie/actie pairs. cluster In the Data ONTAP 7.1 release family, refers to a pair of storage systems (sometimes called nodes) configured to sere data for each other if one of the two systems stops functioning. In Data ONTAP 8.x 7-Mode, a cluster is a group of connected nodes (storage systems) that share a global namespace and that you can manage as a single irtual serer or multiple irtual serers, proiding performance, reliability, and scalability benefits. HA (high aailability) In Data ONTAP 8.x 7-Mode, the recoery capability proided by a pair of nodes (storage systems), called an HA pair, that are configured to sere data for each other if one of the two nodes stops functioning. HA pair In Data ONTAP 8.x 7-Mode, a pair of nodes (storage systems) configured to sere data for each other if one of the two nodes stops functioning. Storage terms ACP Alternate Control Path is a protocol that enables Data ONTAP to manage and control the storage expansion unit storage subsystem using a separate network from the data path, so management communication is not dependent on the data path being intact and aailable. AT-FCX The controller module of the EXN1000 serial adanced technology attachment (SATA) storage expansion unit. Disk Any hard disk drie Disk shelf or shelf Any storage expansion unit containing hard disk dries. ESH The controller module of Fibre Channel disk storage expansion units. IOM The Input/Output module of SAS disk storage expansion units. Loop or Fibre-Channel Arbitrated Loop One or more daisy-chained Fibre Channel storage expansion units connected to an N series storage system. Node The system controller module that executes the software on an N series system. There is one node in single-controller models; there are two nodes in actie/actie or high aailability models. Stack or SAS stack A stack of one or more SAS storage expansion units using IOM modules connected to the storage system. The maximum number of storage expansion units in a stack and the number of storage expansion unit stacks in a configuration are dependent on the type of storage system. xxi IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

25 Command conentions You can enter commands on the system console or from any client that can obtain access to the storage system using a Telnet session. In examples that illustrate commands executed on a UNIX workstation, the command syntax and output might differ, depending on your ersion of UNIX. Formatting conentions The following table lists different character formats used in this guide to set off special information. Formatting conention Type of information Italic type Words or characters that require special attention. Placeholders for information you must supply. For example, if the guide requires you to enter the fctest adaptername command, you enter the characters fctest followed by the actual name of the adapter. Book titles in cross-references. Monospaced font Command and daemon names. Bold monospaced font Information displayed on the system console or other computer monitors. The contents of files. Words or characters you type. What you type is always shown in lowercase letters, unless your program is case-sensitie and uppercase letters are necessary for it to work properly. Keyboard conentions This guide uses capitalization and some abbreiations to refer to the keys on the keyboard. The keys on your keyboard might not be labeled exactly as they are in this guide. What is in this guide... hyphen (-) Enter type enter What it means... Used to separate indiidual keys. For example, Ctrl-D means holding down the Ctrl key while pressing the D key. Used to refer to the key that generates a carriage return, although the key is named Return on some keyboards. Used to mean pressing one or more keys on the keyboard. Used to mean pressing one or more keys and then pressing the Enter key. About this document xx

26 How to send your comments Your feedback helps us to proide the most accurate and high-quality information. If you hae comments or suggestions for improing this document, please send them by to Be sure to include the following information: Exact publication title Publication form number (for example, GC ) Page, table, or illustration numbers A detailed description of any information that should be changed xxi IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

27 Introduction to the EXN3000 storage expansion unit This chapter introduces you to the IBM System Storage EXN3000 SAS/SATA storage expansion unit components, explains EXN3000 storage expansion unit terminology, and shows how the EXN1000/2000/4000 storage expansion units differ. The following topics are discussed: EXN3000 storage expansion unit hardware oeriew Terms and symbols on page 2 Important differences between EXN3000 and EXN1000/2000/4000 storage expansion units on page 4 Information you need to understand about EXN3000 storage expansion units on page 7 Hardware specifications on page 10 EXN3000 storage expansion unit hardware oeriew Figure 1 shows components on the front of the EXN3000 storage expansion unit (sometimes referred to in this and other documents as a shelf or disk shelf). Operator Operation display panel panel Shelf LEDs: - Power - Shelf fault - Actiity Tab for changing the shelf ID Drie Bays Shelf ID digital display (a two digit display) Left mounting tab Up to 24 SAS or SATA disk dries Right mounting tab Drie Bays Disk drie bay numbering 0-23 exn3h016 Figure 1. Front iew of the EXN3000 storage expansion unit The following illustrations show components on the rear of an EXN3000 storage expansion unit. Copyright IBM Corp. 2009,

28 LINK LINK LINK LINK LNK LNK LNK LNK Figure 2 shows an EXN3000 storage expansion unit with two power supplies, PSU 1 and PSU 4 in the proper slots, and blanks in the remaining slots. Power supply LEDs: -power - DC fault - fan fault - AC fault IOM fault LED IOM3 ACP ports SAS ports IOM3 2power supplies, PSU 1 and PSU 4 (each has two integrated fans) 2 I/O blanks 2 power supply blanks, in slots 2 and 3 IOM3 2 I/O modules (IOMs) IOM3 SLOT MAP PSU 1 PSU 3 SBB A Slot A SBB B Slot B PSU 2 PSU 4 exn3h017 Figure 2. Rear iew of the EXN3000 storage expansion unit with two power supplies Figure 3 shows an EXN3000 storage expansion unit with four power supplies. IOM3 IOM3 IOM3 IOM3 exn3h018 4 power supplies Figure 3. Rear iew of the EXN3000 storage expansion unit with four power supplies Terms and symbols Familiarize yourself with the following terms. Many are particular to SAS technology and the EXN3000 storage expansion unit; some are existing terms that also apply. Square and circle symbols: The SAS ports and ACP ports on the storage expansion unit IOMs are designated by square and circle symbols. All cabling is done in reference to connecting to a square port or a circle port. Note: SAS connectiity does not use the concept of In ports and Out ports. 2 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

29 IOM: The SAS shelf I/O Module is located in the back of the storage expansion unit. It connects the indiidual disk dries to the rest of the storage system and controls the storage expansion unit operator display panel LEDs. Each storage expansion unit has two IOMs: IOM A and IOM B. These are also referred to as SBB A and SBB B on the slot map label on the back of the storage expansion unit. SAS stack: Also referred to as stack. A group of one or more SAS disk sheles connected (daisy-chained) together and connected to the controller through the first disk shelf in the stack and the last disk shelf in the stack (as needed). The maximum number of storage expansion units in a stack of storage expansion units and the number of storage expansion unit stacks supported in a configuration are dependent on the type of storage system. ACP: Alternate Control Path is a protocol that enables Data ONTAP to manage and control the disk shelf storage subsystem. It uses a separate network from the data path, so management communication is not dependent on the data path being intact and aailable. Use of ACP requires that all disk shelf IOMs and storage system controllers connect through the ACP ports on the IOMs and the designated network interface on each controller. QSFP: The standard SAS cable connector for all IBM SAS systems. QSFP-to-QSFP SAS cables are used to daisy-chain SAS storage expansion unit ports and connect to the quad-port SAS Host Bus Adapter (HBA) or onboard SAS ports. QSFP to mini-sas cables are used to connect storage expansion units to the dual-port mini-sas HBA. Software-based disk ownership: Software-based disk drie ownership is stored on the disk drie rather than determined by the topology of the storage system's physical connections. It gies you increased flexibility and control oer disk drie use. SAS and SATA disk dries use software-based disk drie ownership. Therefore, the hardware-based disk ownership concept that the local node owns all direct-attached disk dries connected to it through the A channel (the loop attached to the A module on the disk shelf) and its partner owns the disk dries connected to the local node on the B channel, does not apply. This also means that the hardware-based disk drie ownership terms do not apply: primary path, standby path, redundant path, and redundant standby path. Instead, connectiity between the controller and storage expansion unit is addressed in terms of first connections and last connections. First connection: The cabling connection from the controller to the first, or closest, storage expansion unit in a stack of storage expansion units. Last connection: The cabling connection from the controller to the last storage expansion unit in a stack of storage expansion units. Shelf-to-shelf (daisy-chain): The cabling connection between storage expansion units in a stack of more than one storage expansion unit. Each stack of storage expansion units is daisy-chained through its SAS ports and if you are using the ACP capability, each storage expansion unit is also daisy-chained through its ACP ports. Introduction to the EXN3000 storage expansion unit 3

30 Controller: The component of your storage system that runs Data ONTAP software and manages the storage subsystem. The controller located in the top slot in a storage system chassis can be referred to as Controller 1 or Controller A. The controller located in the bottom slot in a storage system chassis can be referred to as Controller 2 or Controller B. Single-controller configuration: A storage system haing one controller. Actie/actie (high aailability) configuration: A storage system haing two controllers. This can be two controllers in one chassis or it can be two controllers, each in a separate chassis. The two controllers sere data for each other if one of the controllers becomes impaired. Multipath HA: Each controller in the actie/actie (high aailability) configuration has multiple ways to connect to a disk drie. Multipath cabling is the most resilient and preferred configuration for actie/actie (high aailability) configurations because it takes full adantage of the resiliency capability of SAS, which means the storage system continues to hae access to disk dries in the eent of cable, HBA, and IOM failure. IOM failures do not result in a controller failoer. Dual-path: The single-controller storage system has multiple ways to connect to a disk drie. This is the preferred configuration for single-controller configurations. Single-path HA: Each controller in the actie/actie (high aailability) storage system has one way to connect to a disk drie. This means an IOM or cable failure requires a controller failoer. Single-path: The single-controller storage system has one way to connect to a disk drie. This configuration has multiple single points of failure and is not a preferred configuration. Configured systems: A new storage system that ships with EXN3000 storage expansion units and preinstalled SAS HBAs (if applicable). Add-on storage expansion units: EXN3000 storage expansion units that are shipped indiidually, and not shipped as part of a configured system. Important differences between EXN3000 and EXN1000/2000/4000 storage expansion units Before you install, cable, and manage SAS expansion units, you should understand the mechanical differences between the SAS storage expansion units (such as EXN3000) and the FC storage expansion units (such as EXN1000/2000/4000), and the infrastructure and resiliency differences between Serial Attached SCSI (SAS) and Fibre Channel (FC) technologies. Mechanical differences The mechanical differences between the SAS and FC storage expansion units are shown in Table 1 on page 5. 4 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

31 Table 1. Mechanical differences between SAS and FC storage expansion units SAS storage expansion units FC storage expansion units Storage expansion units are 4U high Storage expansion units are 3U high. Holds 0 to inch SAS or SATA disk dries. IOM3: Uses 3 Gb/s IOM expanders (each expander has two SAS wide-ports that contain four separate lanes, each with 3 Gb/s of bandwidth). IOM6: Uses 6 Gb/s IOM expanders (each expander has two SAS wide-ports that contain four separate lanes, each with 6 Gb/s of bandwidth). Uses QSFP connectors and utilizes the embedded serial number information, which shows you what cables are connected where. Holds 0 to inch FC-AL disk dries or SATA disk dries. Uses up to 4 Gb/s: 4 Gb: EXN4000/ESH4 2 Gb: EXN2000/ESH/ESH2/ESH4, EXN1000 Uses SFP technology without utilizing the serial number information. Infrastructure differences The infrastructure differences between SAS and FC storage expansion units are shown in Table 2. Table 2. Infrastructure differences between SAS and FC storage expansion units SAS storage expansion units FC storage expansion units A series of storage expansion units in a storage system is called a stack. A series of storage expansion units in a storage system is called a loop. SAS stacks direct point-to point HBA-to-drie connections going only through shelf expanders disk dries are completely isolated from one another. Storage expansion unit ports are designated as squares and circles to signify the point-to-point HBA-to-drie connections. Uses 4x-wide ports linking HBAs to shelf expanders. If one link is down, other links can be used. Supports 15K SAS and 7.2K SATA disk dries in the same stack, but not the same storage expansion unit. SAS and SATA disk dries use software-based disk ownership. Ownership of a disk drie is assigned to a specific storage system by writing software ownership information on the disk drie rather that by using the topography of the storage system s physical connections. Unique storage expansion unit IDs per storage system. (A number from 0 to 98). FC loops are irtual HBA-to-drie connections, passing through all dries on the loop. Storage expansion unit ports are designated as In and Out to signify the directional path of the loop. Uses one physical link linking HBAs to the disk dries. FC and SATA storage expansion units must be in different loops. Also, FC and SATA disk dries cannot be in the same storage expansion unit. The N3700 storage system is the exception. FC and SATA disk dries use hardware-based disk ownership. The local node (first controller) owns all direct-attached disk dries connected to it through the A channel (the loop attached to the A module on the disk shelf). Its partner node (second controller) owns the disk dries connected to the local node on the B channel. Unique storage expansion unit IDs per loop. (A number from 1 to 6). Introduction to the EXN3000 storage expansion unit 5

32 Table 2. Infrastructure differences between SAS and FC storage expansion units (continued) SAS storage expansion units FC storage expansion units You must turn on the power to the storage expansion unit to change the digital display shelf ID. The digital display is on the front of the storage expansion unit. You must power-cycle a storage expansion unit after the shelf ID is changed for it to take effect: Changing the shelf ID on a storage expansion unit that is part of an existing storage system running Data ONTAP requires that you wait at least 30 seconds before turning the power back on so that Data ONTAP can properly delete the old storage expansion unit address and update the copy of the new storage expansion unit address. Changing the shelf ID on a storage expansion unit that is part of a new storage system installation (the storage expansion unit is not yet running Data ONTAP), requires no wait; you can immediately power-cycle the storage expansion unit. The storage expansion unit power can be off when changing the thumb-dial shelf ID. The thumb-dial is on the back of the storage expansion unit. You must power-cycle a storage expansion unit after the shelf ID is changed for it to take effect. Resiliency differences The resiliency differences between SAS and FC storage expansion units are shown in Table 3. Table 3. Resiliency differences between SAS and FC storage expansion units SAS storage expansion units FC storage expansion units Use the Alternate Control Path (ACP) capability: ACP is a protocol that enables Data ONTAP to manage and control the storage expansion unit storage subsystem using a separate network (alternate path) from the data path. Use Serial Attached SCSI (SAS) technology. Do not use the ACP capability. If the FC loop is not operational, there is no access to the entire loop of disk sheles. Manual interention is required. Use FC-AL technology. SAS is a point-to-point disk connection architecture, meaning that the controller can communicate with more than one deice at a time. First and last cabling connections in a storage system allow for suriing any path loss. FC-AL is an arbitrated loop that moes data in one direction. For non actie/actie (non high aailability) multipath pair configurations, cable breaks or storage expansion unit power loss results in downstream disk drie loss. 6 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

33 Information you need to understand about EXN3000 storage expansion units Before you install one or more storage expansion units in a rack or manage existing storage expansion units, you need to understand the following information: Minimum software requirements SAS and SATA disk drie support Power supplies Disk bay numbering Storage expansion unit IDs on page 8 Disk dries on page 9 FC/SAS bridge for stretch or fabric-attached MetroCluster systems on page 9 Minimum software requirements Your storage system must meet the following software requirements: Data ONTAP 7.x releases: Data ONTAP or a later ersion of the 7.x release family N3400 and N3600 storage systems can use Data ONTAP or a later ersion of the 7.x release family. Data ONTAP 8.x releases: Data ONTAP 8.0.2P3 or a later ersion of the 8.x release family N3400 storage systems can use Data ONTAP or a later ersion of the 8.x release family. For a current list and history of Data ONTAP releases aailable by release family and supported storage systems, access the IBM N series support website, as described in Websites on page xxi, and refer to the Data ONTAP Matrix. SAS and SATA disk drie support A single EXN3000 storage expansion unit must contain only SAS disk dries or only SATA disk dries. Howeer, a stack of EXN3000 storage expansion units can contain storage expansion units with SAS disk dries and storage expansion units with SATA disk dries. Power supplies An EXN3000 storage expansion unit requires a different number of power supplies depending on the RPM of your disk dries to insure proper cooling, airflow, and power to your expansion unit. Disk dries that are less than 10K RPM require two power supplies and two power supply blanks to fill the four power supply bays in a storage expansion unit. Disk dries that are 10K RPM or faster, typically SAS disk dries, require four power supplies in a storage expansion unit. Disk bay numbering The EXN3000 storage expansion unit has 24 disk bays, which are numbered 0 through 23. The numbering sequence is displayed on the right mounting tab of the storage expansion unit, as shown in Figure 1 on page 1. Introduction to the EXN3000 storage expansion unit 7

34 Storage expansion unit IDs Unique ID per system: Each SAS storage expansion unit in a storage system must hae a unique ID; a alid shelf ID is from 0 through 98. For N3400 and N3600 storage systems, a alid shelf ID is from 01 through 98. For example, a storage system with two stacks of 10 storage expansion units would hae the storage expansion units numbered from 0 to 19. If your storage system has SAS and FC disk sheles, shelf IDs do not need to be unique between the SAS and FC disk sheles. (FC disk shelf IDs continue to be unique within each FC loop. SAS disk shelf IDs continue to be unique to all other SAS disk sheles in the storage system, including the N3400 and N3600 systems internal shelf ID.) Note: A storage system includes the controller and the storage expansion units. For N3400 and N3600 storage systems, it also includes the internal disk shelf, which is preset to 00 and cannot be changed. The 30-second rule: You must always power-cycle a storage expansion unit when you change the shelf ID. Howeer, when changing the shelf ID on a storage expansion unit that is running Data ONTAP, you must wait at least 30 seconds before turning the power back on. This ensures that Data ONTAP can properly delete the old storage expansion unit address and update the copy of the new storage expansion unit address. Note: A storage expansion unit ID is only considered changed if you select a different storage expansion unit ID since the last power cycle. Configured systems: For new storage systems (configured systems), each storage expansion unit ships with its assigned ID set on its front panel digital display. If you hae more than one storage expansion unit, before cabling your system, do a isual inspection to ensure that the factory set IDs are unique to each storage expansion unit in the storage system. You need to turn on the power to the storage expansion units to do this. Add-on storage expansion units: If you order additional storage expansion units to hot-add to your existing storage system or to replace an existing storage expansion unit, you must set the storage expansion unit ID yourself. Note: For N3400 and N3600 storage systems, internal storage expansion unit IDs are preset to 00 and cannot be changed. IDs for attached storage expansion units must be 01 through 98. Duplicate storage expansion unit IDs: If two or more SAS storage expansion units hae the same ID, the error message, sas.shelf.conflict, appears on the system console and the system assigns a soft ID of a number greater than 100 to the duplicate storage expansion unit. The disk drie addresses use the storage expansion unit serial number. The digital display continues to display the ID you assigned. You must reassign the duplicate shelf ID. 8 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

35 Disk dries For example, if you assigned storage expansion unit ID 25 to two storage expansion units in your storage system, both storage expansion units display 25 on their digital display. Supported disk dries See the IBM System Storage N series Introduction and Planning Guide for information about supported disk dries and storage systems. This guide, as well as other N series documentation, is aailable on the IBM N series support website, which is accessed and naigated as described in Websites on page xxi. Disk drie addressing Disk dries are identified with a triple naming scheme for SAS direct attached disk dries as shown in the following table. Table 4. Disk drie addressing Disk connection Disk name Example SAS direct attached <slot><port>.<shelfid>.<bay> The internal SAS-connected disk in bay 9 for a N3400 or N3600 storage system would hae an address of 0c.0.9. The disk in shelf 2, bay11, connected to onboard port 0a would hae an address of 0a.2.11 The disk in shelf 6, bay 3, connected to an HBA in slot 1, port c, would hae an address of 1c.6.3. Note: If there are two storage expansion units with the same ID, although the system assigns a soft ID of a number greater than 100 to the duplicate storage expansion unit, the disk drie address uses the storage expansion unit serial number. For example, the duplicate storage expansion unit ID for a SAS direct-attached disk connection would be 1c.<storage expansion unit serial number>.3. FC/SAS bridge for stretch or fabric-attached MetroCluster systems Note: The guides referenced here and in the "Supporting FC/SAS bridge configurations", as well as other N series documentation, are aailable on the IBM N series support website, which is accessed and naigated as described in Websites on page xxi. See the IBM System Storage N series Introduction and Planning Guide and ATTO FibreBridge 6500N Installation and Operations Manual for information about FC/SAS bridge hardware. Supporting FC/SAS bridge configurations See Configuring a MetroCluster with SAS disk sheles and FibreBridge 6500N for information on installing, hot-adding, conerting HA pairs to fabric-attached MetroClusters, and replacing FC/SAS bridge. Introduction to the EXN3000 storage expansion unit 9

36 Hardware specifications The following tables list the characteristics and requirements for your EXN3000. Table 5. EXN3000 physical characteristics Weight With maximum number of disk dries Without disk dries Empty Rack units Height Width Depth 110 lbs (49.9 kg) 53.7 lbs (24.4 kg) 21.1 lbs (9.6 kg) 4U 7 in. (17.8 cm) 17.7 in. (45 cm) 24 in. (61 cm) Table 6. EXN3000 clearance dimensions Front (cooling) Rear (cooling and maintenance) Front (maintenance) 6 in. (15.3 cm) 12 in. (30.5 cm) 25 in. (55.95 cm) Table 7. EXN3000 enironmental requirements Operating temperature maximum range 10 C to 40 C (50 F to 104 F) Operating temperature recommended range 20 C to 25 C (68 F to 77 F) Nonoperating temperature range -40 C to 70 C (-40 F to 158 F) Recommended operating range (relatie humidity) 40 to 55% Maximum operating range (relatie humidity) 20 to 80% (non-condensing) Non-operating range (relatie humidity) 10 to 95% (non-condensing). Maximum wet bulb temperature 28 C (82 F) Maximum altitude 3045 m (10,000 ft.) Acoustic leel Idle 5.7 bels (SATA dries) 6.0 bels (SAS dries) Operating 6.7 bels (SATA dries) 7.0 bels (SAS dries) 10 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

37 Table 8. EXN3000 electrical requirements Wet bulb (caloric alue) 2,201 Btu/hr (fully loaded shelf, SAS dries) 1,542 Btu/hr (fully loaded shelf, SATA dries) Maximum electrical power 100 to 240 VAC 16-6 A (8-3A max per inlet); 50/60Hz Nominal Electrical Power 100 to 120 VAC, 6 A; 200 to 240 VAC, 3 A, 50/60 Hz (SAS dries) 100 to 120 VAC, 4.4 A; 200 to 240 VAC, 2.1 A, 50/60 Hz (SATA dries) Table 9. EXN3000 electrical requirements SAS dries 100 to 120V 200 to 240V (200V actual) 200 to 240V (215V actual) Input oltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Size (GB) Worstcase, two PSUs Per PSU pair Typical System, four PSUs Worstcase, two PSUs Per PSU pair Typical System, four PSUs Worstcase, two PSUs Per PSU pair Typical n/a n/a System, four PSUs n/a Table 10. EXN3000 electrical requirements 4TB encrypted and unencrypted SAS disk dries: 100 to 120V 200 to 240V (200V actual) Input oltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Worst-case, two PSUs Per PSU pair Typical System, two PSUs Worst-case, two PSUs Per PSU pair Typical System, two PSUs Note: 1. Worst-case indicates a system running with two PSUs, high fan speed, and power distributed oer two power cords. 2. Per PSU pair indicates typical power needs, per PSU pair, for a system operating under normal conditions. 3. System indicates typical power needs for four PSUs in a system operating under normal conditions and power distributed oer four power cords. Introduction to the EXN3000 storage expansion unit 11

38 4. Depending on the disk drie type, your EXN3000 might hae two or four power supplies. EXN3000s with SAS disk dries hae four power supplies and require the use of four power outlets. Table 11. EXN3000 electrical requirements SATA dries 100 to 120V 200 to 240V (200V actual) 200 to 240V (215V actual) Input oltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Size (GB) Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical TB TB n/a 3 TB (SSD) n/a TB TB n/a 3 TB (SSD) n/a System, two PSUs 500 1, ,501 1, ,501 1, ,542 1 TB 1, ,495 1, ,448 1, ,542 2 TB 1, ,561 1, ,554 n/a 3 TB 1, ,555 1, , (SSD) n/a Note: 1. Worst-case indicates a system running with two PSUs, high fan speed, and power distributed oer two power cords. 2. Per PSU pair indicates typical power needs, per PSU pair, for a system operating under normal conditions. 3. System indicates typical power needs for four PSUs in a system operating under normal conditions and power distributed oer four power cords. 4. Depending on the disk drie type, your EXN3000 might hae two or four power supplies. EXN3000s with SAS disk dries hae four power supplies and require the use of four power outlets. Table 12. Electrical requirements SATA dries, GB or GB disk dries: Input oltage 100 to 120V 200 to 240V (200V actual) Input current measured, A Qty. Worstcase, single PSU Typical Per PSU Sys., two PSUs Worstcase, single PSU Typical Per PSU Sys., two PSUs IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

39 Table 12. Electrical requirements SATA dries, GB or GB disk dries: (continued) Input oltage 100 to 120V 200 to 240V (200V actual) Input power measured, W Thermal dissipation, BTU/hr Qty. Worstcase, single PSU Typical Per PSU Sys., two PSUs Worstcase, single PSU Typical Per PSU Sys., two PSUs Note: 1. Worst-case indicates a system running with two PSUs, high fan speed, and power distributed oer two power cords. 2. Per PSU pair indicates typical power needs, per PSU pair, for a system operating under normal conditions. 3. System indicates typical power needs for four PSUs in a system operating under normal conditions and power distributed oer four power cords. 4. Depending on the disk drie type, your EXN3000 might hae two or four power supplies. EXN3000s with SAS disk dries hae four power supplies and require the use of four power outlets. Table 13. Electrical requirements SATA dries, 6-100GB SSD dries with 18-1 TB or 18-3 TB disk dries: Input oltage 100 to 120V 200 to 240V (200V actual) Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Size (GB) Worstcase, single PSU Typical Per PSU Sys., two PSUs Worstcase, single PSU Typical Per PSU Sys., two PSUs 1 TB TB TB TB TB 1, ,144 1, ,130 3 TB 1, ,256 1, ,355 Note: 1. Worst-case indicates a system running with two PSUs, high fan speed, and power distributed oer two power cords. 2. Per PSU pair indicates typical power needs, per PSU pair, for a system operating under normal conditions. 3. System indicates typical power needs for four PSUs in a system operating under normal conditions and power distributed oer four power cords. 4. Depending on the disk drie type, your EXN3000 might hae two or four power supplies. EXN3000s with SAS disk dries hae four power supplies and require the use of four power outlets. Introduction to the EXN3000 storage expansion unit 13

40 14 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

41 Installing a storage expansion unit for a new storage system Installation safety rules This chapter proides information for installing and cabling an EXN3000 storage expansion unit for a new storage system installation. Note: For information about hot-adding a storage expansion unit, see Hot-adding a storage expansion unit to an existing system on page 23. The following topics are discussed: Installation safety rules Required manuals, tools, and equipment on page 18 Checking shipment package contents on page 18 Roadmap for new storage system installations on page 18 Installing a storage expansion unit for a new storage system installation on page 19 Changing the storage expansion unit ID on page 21 Telecommunication regulatory statement This product may not be certified in your country for connection by any means whatsoeer to interfaces of public telecommunications networks. Further certification may be required by law prior to making any such connection. Contact an IBM representatie or reseller for any questions. Use the following safety rules when installing the storage expansion units: You need to obsere the following rules and restrictions when installing a storage expansion unit in a standard 19-inch (48.26 cm) equipment rack with mounting rails: CAUTION LIFTING HAZARD 110 lbs (49.9 kg) To reduce the weight of the disk shelf, remoe the power supplies and IOMs before lifting. exn32006 CAUTION: Use safe practices when lifting. You must work with two other people. Copyright IBM Corp. 2009,

42 For stability, begin installing storage expansion units in a rack from the lowest position. DANGER Install the heaiest equipment from the bottom up when installing it in a moable rack. This helps preent the rack from tipping oer and seriously injuring you or someone else. When installing storage expansion units in a rack, do not exceed the maximum storage limit for your N series storage system. Fully populated storage expansion units weigh 49.9 kg (110 lbs.). Use three people to lift a storage expansion unit into a rack. To make the storage expansion unit lighter and easier to maneuer, remoe the power supplies and IOMs, noting the location of each component for reinstallation. After the storage expansion unit is installed in the rack, reinsert the power supplies and IOMs. Attention: Do not remoe the disk dries or drie blank coers to reduce the weight. When installing storage expansion units and a storage system into a moable cabinet or rack, install from the bottom up for best stability. A safe lifting height is 20U. Do not lift the enclosure by the handles on the PSUs. They are not designed to take the weight. Ensure that the socket outlets are located near the equipment and are easily accessible since the plug on the power supply cord is used as the main disconnect deice. The rack, when configured with the units, must meet the safety requirements of UL and IEC CAUTION: The optional RJ-45 socket on the I/O module is for Ethernet connection only and must not be connected to a telecommunications network. Power and electrical The enclosure must only be operated from a power supply input oltage range of VAC. The rack design should take into consideration the maximum operating ambient temperature for the unit, which is 40 degrees C. The rack should hae a safe electrical distribution system. It must proide oer-current protection for the unit and must not be oerloaded by the total number of units installed in the rack. When addressing these concerns consideration should be gien to the electrical power consumption rating shown on the nameplate. A safe electrical earth connection must be proided to the power supply cords. This equipment is designed for connection to a grounded outlet. The grounding type plug is an important safety feature. To aoid the risk of electrical shock or damage to the equipment, do not disable this feature. The electrical distribution system must proide a reliable earth connection for each unit in the rack. Proide a suitable power source with electrical oerload protection to meet the requirements laid down in the technical specification. 16 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

43 Each PSU in each unit has an earth leakage current of 1.0mA. The design of the electrical distribution system must take into consideration the total earth leakage current from all the PSUs in all the units. The rack might require labeling with HIGH LEAKAGE CURRENT. Earth connection essential before connecting supply. DANGER When four PSUs are used, to ensure protection against electrical shock caused by HIGH LEAKAGE CURRENT (TOUCH CURRENT), the storage expansion unit must be connected to at least two separate independent sources. This is to ensure a reliable earth connection. Handling static-sensitie deices CAUTION: The storage expansion unit uses electronic components that are sensitie to static electricity. Static discharge from your clothing or other fixtures around you can damage these components. Put on an antistatic ESD strap and grounding leash to free yourself of static electricity before touching any electronic components. Installation requirements Attention: Static electricity can damage electronic deices and your system. To aoid damage, keep static-sensitie deices in their static-protectie packages until you are ready to install them. To reduce the possibility of electrostatic discharge (ESD), obsere the following precautions: Limit your moement. Moement can cause static electricity to build up around you. Handle the deice carefully, holding it by its edges or its frame. Do not touch solder joints, pins, or exposed printed circuitry. Do not leae the deice where others can handle and possibly damage the deice. While the deice is still in its static-protectie package, touch it to an unpainted metal part of the system unit for at least two seconds. This drains static electricity from the package and from your body. Remoe the deice from its package and install it directly into your system unit without setting it down. If it is necessary to set the deice down, place it in its static-protectie package. Do not place the deice on your system unit coer or on a metal table. Take additional care when handling deices during cold weather because heating reduces indoor humidity and increases static electricity. The following requirements must be met when installing a new storage system: Stacks of storage expansion units can only contain the EXN3000 with IOM3 or EXN3000 with IOM6. Note: Mixing IOM3 and IOM6 in the same storage expansion unit is not supported. Installing a storage expansion unit for a new storage system 17

44 Your system must meet the Data ONTAP requirements for installing a new storage system, as described in Minimum software requirements on page 7. Make sure that you hae read and understand the safety information. Required manuals, tools, and equipment In addition to this document, you need the following manuals. These documents are aailable on the IBM N series support website, which is accessed and naigated as described in Websites on page xxi. Installation and Setup Instructions that shipped with your N series system IBM System Storage EXN3000 Installation and Setup Instructions IBM System Storage Uniersal SAS and ACP Cabling Guide IBM System Storage Data ONTAP Actie/Actie Configuration Guide or Clustered Data ONTAP High Aailability Configuration Guide for your ersion of Data ONTAP Data ONTAP Software Setup Guide for your ersion of Data ONTAP Attention: Before you begin your installation, print and complete a configuration worksheet for your storage system to gather the information that the software setup process requires. The configuration worksheet is proided in the Data ONTAP Software Setup Guide for your ersion of Data ONTAP. Required tools and equipment for installation You must supply the following tools and equipment: #2 Phillips screwdrier Flat-bladed screwdrier Antistatic ESD strap and grounding leash CAT6 Ethernet LAN cables (required for ACP cabling), if not ordered from IBM Checking shipment package contents Make sure that your shipment package includes the following items. EXN3000 Installation and Setup Instructions. A storage expansion unit containing the power supplies and disks you ordered. Cables and power cords, as ordered. A rail kit for mounting the storage expansion unit in a standard 19-inch rack may also be included. Roadmap for new storage system installations About this task The following steps list the tasks you must complete and the documents you should hae on hand for EXN3000 storage expansion unit installations for new storage systems. Procedure 1. Install the storage system in a rack if it was not shipped in a system cabinet. See the Installation and Setup Instructions and rack kit installation flyer that came with your storage system for instructions. 18 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

45 2. Install the storage expansion unit. See the EXN3000 Installation and Setup Instructions that came with your storage expansion unit or Installing a storage expansion unit for a new storage system installation for more information. 3. Connect the storage expansion unit to a power source and turn on the power to the storage expansion units. See the EXN3000 Installation and Setup Instructions that came with your storage expansion unit or Installing a storage expansion unit for a new storage system installation for more information. 4. Set the storage expansion unit ID. See EXN3000 Installation and Setup Instructions that came with your storage expansion unit or Changing the storage expansion unit ID on page 21 for more information. 5. Cable the storage system. Cable SAS connections are required. ACP connections are optional. SeeUniersal SAS and ACP Cabling Guidefor more information about cabling your storage system. 6. Connect the storage system to a power source and turn on the power to the storage system. See the Installation and Setup Instructions that came with your storage system for more information. 7. Boot the storage system and begin setup. If applicable, enable ACP capability at setup. For more information, see Data ONTAP Software Setup Guide and Data ONTAP Storage Management Guide. Installing a storage expansion unit for a new storage system installation Before you begin The following requirements must be met when installing a new storage system: Verify that your storage system meets the requirements for the disk sheles you are installing and the type of SAS cables you are using to connect the disk sheles. The SAS cables can be SAS copper, SAS optical, or a mix depending on whether or not your system meets the requirements for using the type of cable. If you are using a mix of SAS copper cables and SAS optical cables, the following rules apply: Shelf-to-shelf connections in a stack must be all SAS copper cables or all SAS optical cables. If the shelf-to-shelf connections are SAS optical cables, the shelf-to-controller connections to that stack must also be SAS optical cables. If the shelf-to-shelf connections are SAS copper cables, the shelf-to-controller connections to that stack can be SAS optical cables or SAS copper cables. SAS optical multimode QSFP-to-QSFP cables can be used for controller-to-shelf and shelf-toshelf connections, and are aailable in lengths up to 50 meters. If you are using SAS optical multimode MPO cables with MPO QSFP modules, the following parameters apply: You can use these cables for controller-to-shelf and shelf-to-shelf connections. The length of a single cable cannot exceed 150 meters for OM4 and 100 meters for OM3. The total end-to-end path (sum of point-to-point paths from the controller to the last shelf) cannot exceed 510 meters. The total path includes the set of breakout cables, patch panels, and inter-panel cables. Installing a storage expansion unit for a new storage system 19

46 If you are using SAS optical multimode breakout cables, the following parameters apply: You can use these cables for controller-to-shelf and shelf-to-shelf connections. If you use multimode breakout cables for a shelf-to-shelf connection, you can only use it once within a stack of disk sheles. You must use SAS optical multimode QSFP-to-QSFP or MPO cables with MPO QSFP modules to connect the remaining shelf-to-shelf connections. The point-to-point (QSFP-to-QSFP) path of any multimode cable cannot exceed 150 meters for OM4 and 100 meters for OM3. The path includes the set of breakout cables, patch panels, and inter-panel cables. The total end-to-end path (sum of point-to-point paths from the controller to the last shelf) cannot exceed 510 meters. The total path includes the set of breakout cables, patch panels, and inter-panel cables. Up to one pair of patch panels can be used in a path. You need to supply the patch panels and inter-panel cables. The inter-panel cables must be the same mode as the SAS optical breakout cable: multimode. You receied a set of QSFP-to-MPO cable modules with each set of SAS optical breakout cables, which you must attach to the MPO end of each SAS optical breakout cable. The breakout cables hae SC, LC, or MTRJ connectors on the opposite end, which connect to a patch panel. You must connect all eight (four pairs) of the SC, LC, or MTRJ breakout connectors to the patch panel. Alternate Control Path (ACP) is supported up to 100 meters (per Ethernet standard). You must hae read and understood the safety information section. About this task To install a storage expansion unit for a new storage system, complete the following steps: Note: EXN3000 storage expansion units do not need to be grounded. Procedure 1. Verify that your storage system meets the minimum software requirements to support the storage expansion unit. See Minimum software requirements on page Following the instructions in EXN3000 Storage Expansion Unit Installation and Setup Instructions, install the IBM rail kit to support your storage expansion unit. Attention: If you are installing multiple storage expansion units, install them from the bottom to the top of the rack for best stability. 3. Following the instructions in the EXN3000 Storage Expansion Unit Installation and Setup Instructions, install and secure the storage expansion unit into the rack. 20 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

47 Attention: Fully populated storage expansion units weigh 49.9 kg (110 lbs.). Use three people to lift a storage expansion unit into a rack. To make the storage expansion unit lighter and easier to maneuer, remoe the power supplies and IOMs, noting the location of each component for reinstallation. Do not remoe the disk dries or drie blank coers to reduce the weight. After the storage expansion unit is installed in the rack, reinsert the power supplies and IOMs. 4. If you are adding multiple storage expansion units, repeat steps 2 on page 20 and 3 on page 20 to install the remaining rail kits and storage expansion units. 5. Connect the power cords to each storage expansion unit and secure them in place with the power cord retainer. Attention: Depending on the disk drie type, your disk shelf might hae two or four power supplies. Connect power supplies in slots 1 and 3 to one power source and power supplies in slots 2 and 4 to a different power source to ensure resiliency. 6. Turn on the power to the storage expansion units, and wait for the disk dries to spin up. Note: a. Disk drie spin-up time differs for storage expansion units with four power supplies and two power supplies. For a storage expansion unit with four power supplies, all disk dries spin up at the same time. For a storage expansion unit with two power supplies, each of the four columns of disk dries spin up at 12-second interals. b. If a storage expansion unit with two power supplies has a faulty power supply, the spin-up time takes an additional 12 seconds and the fault LED on the disk drie in bay 23 illuminates for approximately 12 seconds. 7. Visually inspect the storage expansion unit IDs to ensure that the factory set IDs are unique to each storage expansion unit in the storage system. If you need to change a storage expansion unit ID see Changing the storage expansion unit ID. 8. See the Uniersal SAS and ACP Cabling Guide to cable your storage expansion units. Changing the storage expansion unit ID About this task A unique shelf ID is required for each SAS storage expansion unit within the entire storage system. A alid storage expansion unit ID is 0 through 98. For N3400 and N3600 storage systems, attached storage expansion units must be 01 through 98; the internal shelf ID for the N3400 and N3600 storage systems is preset to 00 and cannot be changed. If your storage system has SAS and FC storage expansion units, storage expansion unit IDs do not need to be unique between the SAS and FC storage expansion units. (FC storage expansion unit IDs continue to be unique with each FC loop. SAS storage expansion unit IDs continue to be unique to all other SAS storage expansion units in the storage system, including the N3400 and N3600 systems internal storage expansion unit ID.) If two or more SAS storage expansion units hae the same ID, the error message sas.shelf.conflict appears on the system console and the system assigns a soft Installing a storage expansion unit for a new storage system 21

48 ID of a number greater than 100 to the duplicate storage expansion unit; the disk drie addresses use the storage expansion unit serial number. For example, a disk drie address is arranged as <slot><port>.<storage expansion unit ID>.<bay>. This means that a duplicate storage expansion unit with a disk in bay 3, connected to an HBA in slot 1, port c, would hae an address of 1c.<storage expansion unit serial number>.3. You must reassign the duplicate storage expansion unit ID. Procedure 1. Turn on the power to the storage expansion unit if it is not already on. 2. Remoe the bezel to locate the small U-shaped tab aboe the LED icons. 3. Change the first number of the storage expansion unit ID by pressing and holding the U-shaped tab until the first number on the digital display flashes, which can take two to three seconds. This actiates the storage expansion unit ID programming mode. Note: If the ID takes longer than two to three seconds to flash, press the button again, making sure to press it in all the way. 4. Press the tab to adance the number by one until you reach the required number from 0 to 9. The first number continues to flash. 5. Change the second number of the storage expansion unit ID by pressing and holding the tab until the second number on the digital display flashes, which can take two to three seconds. The first number on the digital display stops flashing. 6. Press the tab to adance the number by one until you reach the required number from 0 to 9. The second number continues to flash. 7. Lock in the required number and exit the programming mode by pressing and holding the tab until the second number stops flashing, which can take two to three seconds. Both numbers on the digital display start flashing and the fault LED on the operations panel illuminates after about fie seconds, alerting you that the pending storage expansion unit ID has not taken effect yet. 8. Power-cycle the storage expansion unit to make the storage expansion unit ID take effect. Note: a. If you decide to change the storage expansion unit ID later, after your storage system is running Data ONTAP, you must wait at least 30 seconds before turning the power back on to complete the power-cycle. This ensures that Data ONTAP can properly delete the old storage expansion unit address and update the copy of the new storage expansion unit address. b. A storage expansion unit ID is only considered changed if you select a different ID since the last power cycle. 9. Reinstall the bezel. 10. Repeat steps 1 through 9 for each additional storage expansion unit. 22 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

49 Hot-adding a storage expansion unit to an existing system You can hot-add a EXN3000 storage expansion unit to an existing stack of EXN3000 storage expansion units, to a SAS HBA, or to an onboard SAS port. Hot-adding a storage expansion unit inoles installing, cabling, and erifying the disk drie and disk storage expansion unit ersions. Requirements for hot-adding storage expansion units The following requirements must be met when hot-adding a storage expansion unit: The EXN3000 storage expansion unit with IOM3 modules can only be hot-added to a stack of EXN3000 storage expansion units with IOM3. The EXN3000 storage expansion unit with IOM6 modules can only be hot-added to a stack of EXN3000 storage expansion units with IOM6. Note: Mixing IOM3 and IOM6 in the same storage expansion unit is not supported. Your system must meet the Data ONTAP requirements for hot-adding storage expansion units, as described in Minimum software requirements on page 7. If you are hot-adding a storage expansion unit to an existing SAS HBA or onboard SAS port, your storage system must hae an aailable but unused PCI SAS HBA or onboard SAS port. If you are hot-adding a storage expansion unit to an existing SAS HBA or onboard SAS port, you must hae completed the SAS cabling worksheet, which is proided in the Uniersal SAS and ACP Cabling Guide, so that you know how to cable your storage expansion unit to the controller. See the Uniersal SAS and ACP Cabling Guide for additional information about cabling your storage system. Error messages The following error messages appear on your storage system console if your attempt at hot-adding the storage expansion unit is unsuccessful. Table 14. Hot-adding error messages Error message ses.shelf.inalnum -- WARNING This message occurs when Data ONTAP detects that a Serial Attached SCSI storage expansion unit connected to the system has an inalid storage expansion unit number. ses.shelf.samenumreassign -- WARNING This message occurs when Data ONTAP detects more than one Serial Attached SCSI (SAS) storage expansion unit connected to the same adapter with the same storage expansion unit number. Correctie action 1. Power-cycle the storage expansion unit. 2. If the problem persists, replace the storage expansion unit modules. 3. If the problem persists, replace the storage expansion unit. 1. Change the storage expansion unit number on the storage expansion unit to one that does not conflict with other storage expansion units attached to the same adapter. Halt the system and reboot the storage expansion unit. 2. If the problem persists, contact technical support. Copyright IBM Corp. 2009,

50 Hot-adding tasks To hot-add a storage expansion unit to a stack of storage expansion units, an existing SAS HBA, or an onboard SAS port, you must complete the following tasks: Installing a storage expansion unit for a hot-add Cabling a storage expansion unit on page 26 Verifying the disk drie, storage expansion unit, and ACP firmware ersions on page 31 Installing a storage expansion unit for a hot-add Before you begin The following requirements must be met when installing a new storage system: Verify that your storage system meets the requirements for the disk sheles you are installing and the type of SAS cables you are using to connect the disk sheles. The SAS cables can be SAS copper, SAS optical, or a mix depending on whether or not your system meets the requirements for using the type of cable. If you are using a mix of SAS copper cables and SAS optical cables, the following rules apply: Shelf-to-shelf connections in a stack must be all SAS copper cables or all SAS optical cables. If the shelf-to-shelf connections are SAS optical cables, the shelf-to-controller connections to that stack must also be SAS optical cables. If the shelf-to-shelf connections are SAS copper cables, the shelf-to-controller connections to that stack can be SAS optical cables or SAS copper cables. SAS optical multimode QSFP-to-QSFP cables can be used for controller-to-shelf and shelf-toshelf connections, and are aailable in lengths up to 50 meters. If you are using SAS optical multimode MPO cables with MPO QSFP modules, the following parameters apply: You can use these cables for controller-to-shelf and shelf-to-shelf connections. The length of a single cable cannot exceed 150 meters for OM4 and 100 meters for OM3. The total end-to-end path (sum of point-to-point paths from the controller to the last shelf) cannot exceed 510 meters. The total path includes the set of breakout cables, patch panels, and inter-panel cables. If you are using SAS optical multimode breakout cables, the following parameters apply: You can use these cables for controller-to-shelf and shelf-to-shelf connections. If you use multimode breakout cables for a shelf-to-shelf connection, you can only use it once within a stack of disk sheles. You must use SAS optical multimode QSFP-to-QSFP or MPO cables with MPO QSFP modules to connect the remaining shelf-to-shelf connections. The point-to-point (QSFP-to-QSFP) path of any multimode cable cannot exceed 150 meters for OM4 and 100 meters for OM3. 24 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

51 The path includes the set of breakout cables, patch panels, and inter-panel cables. The total end-to-end path (sum of point-to-point paths from the controller to the last shelf) cannot exceed 510 meters. The total path includes the set of breakout cables, patch panels, and inter-panel cables. Up to one pair of patch panels can be used in a path. You need to supply the patch panels and inter-panel cables. The inter-panel cables must be the same mode as the SAS optical breakout cable: multimode. You receied a set of QSFP-to-MPO cable modules with each set of SAS optical breakout cables, which you must attach to the MPO end of each SAS optical breakout cable. The breakout cables hae SC, LC, or MTRJ connectors on the opposite end, which connect to a patch panel. You must connect all eight (four pairs) of the SC, LC, or MTRJ breakout connectors to the patch panel. Alternate Control Path (ACP) is supported up to 100 meters (per Ethernet standard). You must hae read and understood the safety information section. About this task To hot-add one or more storage expansion units, you must first install the storage expansion unit. Complete the following steps to install a storage expansion unit: 1. Verify that the storage system meets the minimum software requirements to support the storage expansion unit. See Minimum software requirements on page Following the instructions in the EXN3000 Storage Expansion Unit Installation and Setup Instructions, install the IBM rail kit to support the storage expansion unit. 3. Install and secure the storage expansion unit into the rack. Attention: Fully populated storage expansion units weigh 50 kg (110 lbs.). Use three people to lift a storage expansion unit into a rack. To make the storage expansion unit lighter and easier to maneuer, remoe the power supplies and IOMs, noting the location of each component for reinstallation. Do not remoe the disk dries or drie blank coers to reduce the weight. After the storage expansion unit is installed in the rack, reinsert the power supplies and IOMs. 4. If you are adding multiple storage expansion units, repeat steps 2 and 3 to install the remaining rack kits and storage expansion units. 5. Connect the power cords to each storage expansion unit and secure them in place with the power cord retainer. Note: a. Do not cable the storage expansion unit to the storage system yet. b. Depending on the disk drie type, your storage expansion unit might hae two or four power supplies. c. EXN3000 storage expansion units do not need additional grounding. 6. Turn on the power to the storage expansion units, and wait for the disk dries to spin up. Hot-adding a storage expansion unit to an existing system 25

52 Note: a. Disk drie spin-up time differs for storage expansion units with four power supplies and two power supplies. For a storage expansion unit with four power supplies, all disk dries spin up at the same time. For a storage expansion unit with two power supplies, each of the four columns of disk dries spin up at 12-second interals. b. If a storage expansion unit with two power supplies has a faulty power supply, the spin-up time takes an additional 12 seconds and the fault LED on the disk drie in bay 23 illuminates for approximately 12 seconds. 7. Change the storage expansion unit ID to a alid ID from 0 through 98 that is unique from the other storage expansion units in the storage system. For instructions on how to change the storage expansion unit ID, see the EXN3000 Installation and Setup Instructions that came with your storage expansion unit or Changing the storage expansion unit ID on page 21. For storage systems with multiple storage expansion units, you can erify IDs already in use by entering the following command at the console: sasadmin shelf<adapter ID> For 7-Mode HA pairs, you can enter this command from either node. For clustered systems, you must run this command from the nodeshell of the target node. a. Change the shelf ID to a alid ID that is unique from the other SAS disk sheles in the storage system. b. Power-cycle the disk shelf to make the shelf ID take effect. See Changing the storage expansion unit ID on page 21 for more detailed instructions. 8. If you are not currently using the ACP capability but want to, enable ACP on your storage system by completing the following steps; otherwise, go to Cabling a storage expansion unit Note: It is recommended that you use ACP. a. Enable ACP on the storage system by entering the following command at the system console: options acp.enabled on For 7-Mode HA pairs, you can run this command from either node. For clustered systems, you must run this command from the nodeshell of the target node. b. Dedicate a network interface and specify other parameters, such as network domain and netmask, as prompted. For 7-Mode HA pairs, you must dedicate a network interface for each node. Note: Some storage systems hae a dedicated Ethernet port for ACP that you should use. For example, N3400 systems would use the dedicated e0p port. 9. Go to Cabling a storage expansion unit. Cabling a storage expansion unit To cable a hot-added storage expansion unit to an existing stack of storage expansion units or to an existing SAS HBA, see the following topics: Hot-adding a storage expansion unit to an existing stack of storage expansion units on page IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

53 Hot-adding a storage expansion unit to an existing SAS HBA or onboard SAS port on page 29 Hot-adding a storage expansion unit to an existing stack of storage expansion units About this task Cabling the hot-added storage expansion unit inoles cabling the SAS connections and assigning disk drie ownership, and if applicable, enabling ACP and cabling the ACP connections. Note: This procedure assumes that you are hot-adding a storage expansion unit to the last unit in a stack of storage expansion units. The physical arrangement of your stack might be different. Procedure 1. If you hae not already done so, see Installing a storage expansion unit for a hot-add on page 24 to install your storage expansion unit, ground yourself to the storage system chassis, power on the storage expansion unit, and change the storage expansion unit ID. 2. The next step depends on whether you are using the ACP capability. If you are not currently using the ACP capability but would like to, complete the following steps: a. Enable ACP on the storage system by entering the following command at the console: options acp.enabled on b. Dedicate a network interface and specify other parameters, such as network domain and netmask as prompted. Note: For actie/actie (high aailability) configurations, you must dedicate a network interface for each node. Some storage systems hae a dedicated Ethernet port for ACP that you should use. For example, for N3400 systems you should use the dedicated e0p port. If you are currently using ACP capability, or you do not wish to use ACP, go to step 3. Note: It is recommended that you use ACP. 3. If you hae an N3400 storage system, go to step 5 on page 28; otherwise, disconnect any SAS cables from the last storage expansion unit that are connected to the storage system controllers. Note: a. Leae the daisy-chained cables that connect the storage expansion units together. b. When you disconnect the SAS cables from the last storage expansion unit that is connected to the storage system controller(s), your storage system maintains connectiity through the first connection from the controller(s) and the daisy-chained cables to the last storage expansion unit. 4. Reconnect the SAS cables that you disconnected in step 3 to the same SAS ports in the new storage expansion unit. Hot-adding a storage expansion unit to an existing system 27

54 5. Daisy-chain the SAS ports of the last storage expansion unit to the new storage expansion unit using the SAS cables that came with the new storage expansion unit. Note: The storage system recognizes the new storage expansion unit as soon as all of the disk dries hae spun-up. 6. Verify SAS connectiity. In actie/actie (high aailability) configurations, you can run these commands from either node. For 7-Mode HA pairs, you can run these commands from the system console of either node. For clustered systems, you must run these commands from the nodeshell of the target node. a. Enter the following command to find out what the adapter name is: sasadmin expander_map b. Enter the following command to erify that all disk dries can be seen by the system: sasadmin shelf adapter_name The system displays a representation of your disk shelf populated with all the disk dries it sees. c. Enter the following command to erify that all IOMs (expanders) can be seen by the system (SAS channels and controller ports): sasadmin expander_map adapter_name The following example of output from this command shows that a single expander, IOM B (slot B), in shelf 3 (ID 3) is attached to port 4a (channel 4a) on the controller: Expanders on channel 4a: Leel 1: WWN 500a c3f, ID 3, Serial Number 1006SZ00196, Product DS224IOM6, Re 0134, Slot B 7. Assign disk ownership. For actie/actie (high aailability) configurations, you can run these commands from the system console of either node. Note: The following substeps are for basic disk ownership assignment. For options to specify the disk dries to be assigned or the system to own the disk dries, see the Data ONTAP Storage Management Guide. For 7-Mode HA pairs, you can run these commands from the system console of either node. Note: For clustered systems, see the Clustered Data ONTAP Physical Storage Management Guide for disk drie ownership information on the N series support website (accessed and naigated as described in Websites on page xxi. a. Enter the following command to see all of the unassigned disk dries: disk show -n Result: The disk dries in the hot-added storage expansion unit will hae the ownership state of Not Owned. b. Enter the following command to assign ownership of the disk dries: disk assign all Note: If you need to assign the disk dries to a non-local storage system, you can specify a storage system by using the -o owner_name option. c. Enter the following command to erify the assignments that you made: disk show - 28 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

55 8. The next step depends on whether you are using the ACP capability. If you are using the ACP capability, then complete the following steps: a. Disconnect any intrastack, stack-to-stack and controller-to-storage expansion unit ACP cables from the last storage expansion unit and reconnect them to the same ACP ports in the new storage expansion unit. b. Daisy-chain the ACP ports from the last storage expansion unit to the new storage expansion unit using the Ethernet cables that came with the new storage expansion unit. If you are not using the ACP capability, go to step Verify that all cables are securely fastened. Note: The storage system recognizes the new storage expansion unit as soon as all of the disk dries hae spun-up. 10. If you hot-added a disk shelf with SAS optical cables to a stack of disk sheles connected with SAS copper cables, replace the SAS copper cables for the rest of the shelf-to-shelf connections and the shelf-to-controller connections so that the stack meets the cabling rules stated in the Hotadding a SAS disk shelf section of this document; otherwise, go to the next step. Use the appropriate cable replacement procedure for your configuration in the Replacing SAS Cables section of this document. 11. Go to Verifying the disk drie, storage expansion unit, and ACP firmware ersions on page 31. Hot-adding a storage expansion unit to an existing SAS HBA or onboard SAS port The following requirements must be met for this procedure: Your storage system must hae Data ONTAP or a later 7.3.x ersion or Data ONTAP 8.0 or a later ersion of the 8.x release family. Howeer, it is recommended that you update your storage system to Data ONTAP 7.3.3P2 or a later ersion of the 7.x release family, or Data ONTAP 8.0P1 or a later ersion of the 8.x release family. Your storage system must hae an aailable but unused PCI SAS HBA or onboard SAS port. You must hae already completed the SAS Cabling worksheet in the Uniersal SAS and ACP Cabling Guide. Cabling a storage expansion unit to an existing SAS HBA or onboard SAS port About this task To cable a storage expansion unit to an existing SAS HBA or onboard SAS port, complete the following steps: 1. If you hae not already done so, see Installing a storage expansion unit for a hot-add on page 24 to install your storage expansion unit, ground yourself to the storage system chassis, power on the storage expansion unit, and change the storage expansion unit ID. 2. The next step depends on whether you are using the ACP capability. If you are using the ACP capability, then complete the following steps: a. Enable ACP on the storage system by entering the following command at the console: Hot-adding a storage expansion unit to an existing system 29

56 options acp.enabled on b. Dedicate a network interface and specify other parameters, such as network domain and netmask as prompted. Note: N3400 storage systems use onboard Ethernet port e0p for the network interface. If you are not using the ACP capability, then go to step 3. Note: It is recommended that you use the ACP capability. 3. Use the Cabling SAS ports procedure in the Uniersal SAS and ACP Cabling Guide. This procedure requires that you use your completed SAS cabling worksheet. 4. Verify that all SAS cables are securely attached. Note: The storage system recognizes the new storage expansion unit as soon as all of the disk dries hae spun-up. 5. Verify SAS connectiity. In actie/actie (high aailability) configurations, you can run these commands from either node. For 7-Mode HA pairs, you can run these commands from the system console of either node. For clustered systems, you must run these commands from the nodeshell of the target node. a. Enter the following command to find out what the adapter name is: sasadmin expander_map b. Enter the following command to erify that all disk dries can be seen by the system: sasadmin shelf adapter_name The system displays a representation of your disk shelf populated with all the disk dries it sees. c. Enter the following command to erify that all IOMs (expanders) can be seen by the system (SAS channels/controller ports): sasadmin expander_map adapter_name The following example of output from this command shows that a single expander, IOM B (slot B), in shelf 3 (ID 3) is attached to port 4a (channel 4a) on the controller: Expanders on channel 4a: Leel 1: WWN 500a c3f, ID 3, Serial Number 1006SZ00196, Product DS224IOM6, Re 0134, Slot B 6. Assign disk ownership. For actie/actie (high aailability) configurations, you can run these commands from the system console of either node. Note: The following substeps are for basic disk ownership assignment. For options to specify the disk dries to be assigned or the system to own the disk dries, see the Data ONTAP Storage Management Guide. Note: For clustered systems, see the Clustered Data ONTAP Physical Storage Management Guide for disk drie ownership information on the N series support website (accessed and naigated as described in Websites on page xxi. a. Enter the following command to see all of the unassigned disk dries: disk show -n 30 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

57 Result: the disk dries in the hot-added storage expansion unit will hae the ownership state of Not Owned. b. Enter the following command to assign ownership of the disk dries: disk assign all Note: If you need to assign the disk dries to a non-local storage system, you can specify a storage system by using the -o owner_name option. c. Enter the following command to erify the assignments that you made: disk show - 7. Cable the ACP connections for the new disk shelf using the ACP cabling procedure in the Uniersal SAS and ACP Cabling Guide. 8. Verify that all ACP cables are securely attached. 9. Go to Verifying the disk drie, storage expansion unit, and ACP firmware ersions. Verifying the disk drie, storage expansion unit, and ACP firmware ersions About this task Because Data ONTAP does not always automatically update the disk drie, storage expansion unit, and ACP firmware on hot-added storage expansion units, you must erify that the disk drie, storage expansion unit, and if applicable, ACP firmware are the most current ersions. If they are not, you must manually update the firmware. This procedure is for 7-Mode configurations. For clustered systems, see Clustered Data ONTAP Upgrade and Reert/Downgrade Guide Storage expansion units connected with SAS optical cables require a ersion of disk shelf firmware that supports SAS optical cables. Best practice is to update all storage expansion units disk sheles in the storage system with the latest ersion of disk shelf firmware. Note: Do not reert disk shelf firmware to a ersion that does not support SAS optical cables. Attention: If you discoer that you must update the disk drie firmware, the command you must run to do this, disk_fw_update, can affect I/O on the disk dries on which you are updating the firmware. 1. Verify that the disk drie firmware is the most current ersion: a. Enter the applicable command: For 7-Mode, enter the following command at the system console: sasadmin expander_map For HA pairs, you can run this command on either node. For clustered systems, enter the following command at the clustershell prompt: run -node node_name -command sasadmin expander_map Hot-adding a storage expansion unit to an existing system 31

58 b. Locate the disk shelf firmware information for the hot-added disk shelf in the output. Example 0151 is the disk shelf firmware ersion for shelf number one (Slot A/IOM A) in the storage system: Expanders on channel 4a: Leel 3: WWN 500a ff, ID 1, Serial Number SHU G114C, Product DS424IOM6, Re 0151, Slot A c. Compare the firmware information in the command output with the disk shelf firmware information at nseries to determine the most current disk shelf firmware ersion. 2. If the command output is Downre: no, go to step 3; otherwise update the disk drie firmware: a. Download the latest disk drie firmware by accessing the IBM N series support website, as described in Websites on page xxi. Refer to the list of current HDD firmware ersions and Data ONTAP Disk Qualification Packages (DQPs) and the instructions found in the Hard Disk Drie (HDD) Firmware Matrix. b. Enter the following command at the storage system console: disk_fw_update Attention: Running this command can affect I/O on the disk dries on which you are updating the firmware. Note: For actie/actie (high aailability) pair configurations, you must run this command on both nodes. 3. Verify that the storage expansion unit firmware is the most current ersion: a. Enter the following command at the storage system console: sysconfig - b. Locate the storage expansion unit firmware information for the hot-added storage expansion unit in the output. For example, 0110 is the storage expansion unit firmware ersion for storage expansion unit number one in the storage system: IOM3: shelf 1: IOM3 Firmware re. IOM3 A: 0110 IOM3 B: 0110 IOM6: shelf 1: IOM6 Firmware re. IOM6 A: 0110 IOM6 B: 0110 c. Determine whether the storage expansion unit firmware is the most current ersion by comparing the firmware information in the sysconfig - output with the storage expansion unit firmware information on the IBM N series support website. You can find the most current storage expansion unit firmware ersion by accessing the IBM N series support website, as described in Websites on page xxi, and referring to the list of current Disk Shelf firmware ersions found in the System, RLM, SP, and Disk Shelf Firmware Matrix. 4. The next step depends on whether the firmware ersion in the command output is the most current ersion. If the storage expansion unit firmware ersion displayed in the sysconfig - output is the same as the most current ersion on the IBM N series support website, then no storage expansion unit firmware update is needed. 32 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

59 If the storage expansion unit firmware ersion displayed in the sysconfig - output is an earlier ersion than the most current ersion on the IBM N series support website, then download the latest disk shelf firmware file following the instructions found in the System, RLM, SP, and Disk Shelf Firmware Matrix. Note: For actie/actie or high aailability configurations, you can run the command from either node. 5. If you are using the ACP functionality, erify that the ACP firmware is the most current ersion by completing the following substeps: a. Enter the following command at the system console: storage show acp b. Locate the ACP firmware information for the hot-added storage expansion unit in the output. c. Compare the firmware information in the storage show acp output with the most current ACP firmware information on the IBM N series support website, which is accessed and naigated as described in Websites on page xxi. Refer to the list of the current ACP firmware ersions found in the System, RLM, SP, and Disk Shelf Firmware Matrix. 6. The next step depends on how current the ACP firmware is in the storage show ACP output. If the ACP firmware ersion in the storage shows ACP output that is the same as the most current ersion on the IBM N series support website, then no ACP firmware update is needed. If the ACP firmware ersion in the storage shows ACP output that is an earlier ersion than the most current ersion on the IBM N series support website, then: a. Download the most recent ACP firmware from the IBM N series support website using the instructions found in the System, RLM, SP, and Disk Shelf Firmware Matrix. b. Follow the instructions on the IBM N series support website to extract your firmware files to the /etc/acpp_fw directory in the root olume of your storage system. c. Enter the following command to update the ACP firmware: storage download acp d. Enter the following command to erify the new ACP firmware: storage show acp e. Reissue the storage show acp command once the update has completed to see the new ACP firmware ersion listed in the output. Hot-adding a storage expansion unit to an existing system 33

60 34 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

61 Managing the storage expansion unit This chapter identifies the location and status conditions of the LEDs on the storage expansion unit, and describes how to monitor the storage expansion unit from the error messages displayed on the storage system console and what correctie actions you can take. The following topics are discussed: Storage expansion unit management safety rules Monitoring storage expansion unit components on page 36 Replacing storage expansion unit components on page 44 Storage expansion unit management safety rules Use the following safety rules when operating, replacing, or remoing the storage expansion unit or storage expansion unit components: In order to comply with applicable safety, emission, and thermal requirements, no coers should be remoed and all bays must be populated with plug-in modules. Since all plug-in modules and blank plates are part of the fire enclosure, they must only be remoed when a replacement can be immediately added. The system must not be run without all modules or blanks in place. To reduce the risk of personal injury or equipment damage, allow internal components time to cool before touching them. When remoing a storage expansion unit from a moable cabinet or a rack, empty from the top down. To aoid danger of the rack toppling oer, do not slide more than one enclosure out of the rack at a time. Permanently unplug the unit before you moe it or if you think it has become damaged in any way. The power connection should always be disconnected prior to remoal of a PSU from the enclosure. DANGER Do not remoe a faulty PSU unless you hae a replacement PSU of the correct type ready for insertion. The system must not be run without all units or module blanks in place. DANGER Do not remoe coers from the PSU. Danger of electrical shock inside. Return the PSU to your supplier for repair. CAUTION: This unit has more than one power supply cord. To reduce the risk of electrical shock, disconnect all power supply cords for complete isolation before sericing. Copyright IBM Corp. 2009,

62 Monitoring storage expansion unit components This section discusses the following topics: Monitoring the front operator display panel Monitoring the disk drie on page 37 Monitoring the IOMs on page 39 Monitoring the power supply on page 41 Monitoring the front operator display panel The front left panel of the storage expansion unit has three LED indicators, which indicate whether your storage expansion unit is functioning normally or there are problems with the hardware. The following illustration shows the location of the operator display panel LEDs and storage expansion unit ID digital display. Power Shelf fault Actiity exn3h071 Figure 4. Operator display panel LEDs The following table describes the meaning of the operator display panel LEDs when illuminated. To identify any hardware failure associated with the operator display panel, see Operator display panel console error messages on page 37 and the error messages displayed on your storage system console. Table 15. Operator display panel LEDs Icon LED Color When illuminated Power Green One or more power supplies are supplying AC power to the storage expansion unit. IOMs and power supplies are functioning normally. 36 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

63 Table 15. Operator display panel LEDs (continued) Icon LED Color When illuminated Storage expansion unit fault Amber An error occurred with the function of a disk drie, IOM, or power supply. Actiity Green A link is established between the storage expansion unit and the storage system or another storage expansion unit that it is attached to. Operator display panel console error messages The following error messages appear on your storage system console if a SCSI Enclosure Serices (SES) process on the storage expansion unit detects that the front operator display panel failed, or that there is a problem with the storage expansion unit or its components. For information about replacing an IOM, see Hot-swapping or replacing an IOM on page 51. Table 16. Operator display panel console error messages Error message Action required ses.status.displayerror This message occurs when the SCSI Enclosure Serices (SES) module in the storage expansion unit detects an error in the storage expansion unit display panel. The storage expansion unit might be unable to proide correct addresses to its disks. 1. If possible, erify that the connection between the storage expansion unit and the display is secure. 2. Verify that the SES module or modules are fully seated; replacing them might sole the problem. 3. If the problem persists, the SES module that detected the warning condition might be faulty. 4. If the problem persists after the module or modules are replaced, replace the storage expansion unit. 5. If the problem persists, contact technical support. Monitoring storage expansion unit components Monitoring the disk drie The disk drie has two LED indicators on the front right side, which indicate whether the disk drie is functioning normally or there are problems with the hardware. The following illustration shows the front of the disk drie and the location of the two LEDs. Managing the storage expansion unit 37

64 Actiity 300GB exn3h075 Fault Figure 5. Disk drie LED indicators The following table describes the meaning of the disk drie LEDs when illuminated. To identify any hardware failure associated with a disk drie, see Disk drie console error messages and the error messages displayed on your storage system console. Table 17. Disk drie LEDs LED Color When illuminated Actiity Green Solid green: the disk drie has power. Flashing green: the disk drie has power and I/O is in progress. Fault Amber Solid amber: there is an error with the function of the disk drie. Disk drie console error messages The following error messages appear on your storage system console if an SES element on the disk drie fails, if solid state dries and rotating dries are in the same SAS domain, or if solid state drie consumption thresholds are reached. For information about replacing disk dries, see Hot-swapping a disk drie on page 49. Table 18. Disk drie console error messages Error message ses.status.drieerror -- NODE_ERROR This message occurs when a critical condition is detected for the disk drie in the storage expansion unit. The disk drie might fail. Action required 1. Make sure that the disk drie is not running on a degraded olume. If it is, then add as many spares as necessary into the system, up to the specified leel. 2. After the olume is no longer in degraded mode, replace the disk drie that is failing. 38 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

65 LNK LNK Table 18. Disk drie console error messages (continued) Error message Action required ses.shelf.ctrlfailerr -- SVC_ERROR This message occurs when the adapter and loop ID of the SCSI Enclosure Serices (SES) target for which the SES has control fail. shm.threshold.spareblocksconsumedmax This warning message occurs when a specified number of dries hae consumed at least 80 percent of their use-based internal spare capacity. The affected dries are also identified. config.mixedsasdomain This warning message occurs when the identified channel has solid state dries and rotating dries connected in the same SAS domain. shm.threshold.spareblocksconsumed This notification message occurs when a specified number of dries hae consumed at least 60 percent of their use-based internal spare capacity. The affected dries are also identified. 1. Check the LEDs on the storage expansion unit and the storage expansion unit modules on the back of the storage expansion unit to see whether there are any abnormalities. If the modules appear to be problematic, replace the applicable module. 2. If the SES target is a disk drie, check to see whether the disk drie failed. If it failed, replace the disk drie. Ensure that replacement dries are aailable, but do not initiate drie replacement. The dries will be failed by Data ONTAP if appropriate. Solid state dries and rotating dries must be configured in different SAS domains. Plan for solid state drie replacement. Monitoring the IOMs Each storage expansion unit IOM has three LED indicators, which indicate whether the IOM is functioning normally, the IOM's readiness for I/O traffic, or whether there are any problems with the hardware. The following illustration shows the locations of the IOM LEDs. Note: IOM3 appears on the IOM face for EXN3000 storage expansion units containing IOM3. IOM6 appears on the face of the IOM for EXN3000 storage expansion units containing IOM6. Fault LNK IOM3 IOM3 exn3h076 Figure 6. IOM LED indicators Managing the storage expansion unit 39

66 The following table describes the meaning of the IOM LEDs when illuminated. To identify any hardware failure associated with an IOM, see IOM console error messages and the error messages displayed on your storage system console. Note: At power up, the fault LED on the IOM illuminates by default. After a good operating condition is established, the fault LED turns off and the LNK LED illuminates. Table 19. IOM LEDs Icon LED Color When illuminated Fault Amber An error occurred with the function of the IOM. LNK Link Green A communication path with other storage expansion units and the storage system is established. IOM console error messages The following error messages appear on the storage system console if an SES element on the IOM fails. For more information about hot-swapping the IOM, see Hot-swapping or replacing an IOM on page 51. Table 20. IOM console error messages Error message ses.status.electronicserror This message occurs when a failure has been detected in the module that proides disk SCSI Enclosure Serices (SES) monitoring capability. ses.status.moduleerror This message occurs when the reporting storage expansion unit detects an error in the indicated storage expansion unit module. ses.status.acperror This message occurs when a failure has been detected in the ACP processor that proides alternate control path capability. ses.status.acpwarn Correctie action Replace the module. In some storage expansion unit types, this function is integrated into the Fibre Channel, SCSI, or Serial Attached SCSI (SAS) interface modules. 1. Verify that the storage expansion unit module is fully seated and secure. 2. If the problem persists, replace the storage expansion unit module. Replace the module. In some storage expansion unit types, this function is integrated into the SAS interface modules. This message occurs when a non-fatal condition has been detected in the ACP processor that proides alternate control path capability. 40 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

67 Monitoring the power supply An EXN3000 storage expansion unit requires a different number of power supplies depending on the RPM of your disk dries to insure proper cooling, airflow, and power to your storage expansion units: Disk dries that are less than 10K RPM require two power supplies and two power supply blanks to fill the four power supply bays in a storage expansion unit. Disk dries that are 10K RPM or faster, typically SAS disk dries, require four power supplies in a storage expansion unit. See Figure 2 on page 2 and Figure 3 on page 2 for illustrations of the storage expansion unit with two power supplies and four power supplies. Power supplies are oriented so that the cam handle release latch always faces toward the chassis center, and the LEDs, power switch, and power plug always face away from the chassis center. Each power supply unit has four LEDs on the back, which indicate whether the power supply or the two integrated fan modules are functioning normally, or there are problems with the hardware. The following illustration shows the location of the power supply LEDs. Figure 7. Power supply unit LEDs The following table describes the meaning of the power supply LEDs when illuminated. To identify any hardware failure associated with a power supply, see Power supply console error messages on page 42 and the error messages displayed on your storage system console. Managing the storage expansion unit 41

68 Table 21. Power supply LEDs Icon LED Color When illuminated PSU OK Green Power supply is functioning normally. Note: The other three LEDs are not illuminated. Fan fault Amber An error occurred with the function of a fan. AC fault Amber The power supply is not turned on or the AC power cord in not plugged in. Power supply console error messages The following error messages appear on your storage system console if an SES element on the power supply fails. For information about replacing the power supply, see Replacing a power supply on page 53. Table 22. Power supply console error messages Error message ses.status.pserror This message occurs when a critical condition is detected in the indicated storage expansion unit power supply. The power supply might fail. ses.status.fanerror This message occurs when the indicated storage expansion unit, cooling fan, or fan module fails, and the storage expansion unit or its components are not receiing required cooling airflow. Correctie action 1. Verify that power input to the storage expansion unit is correct. If separate eents of this type are reported simultaneously, the common power distribution point might be at fault. 2. If the storage expansion unit is in a cabinet, erify that the power distribution unit is ON and functioning properly. Make sure that the storage expansion unit power cords are fully inserted and secured, the supply is fully seated and secured, and the supply is switched ON. 3. Verify that power supply fans are functioning. If the problem persists, replace the power supply. 4. If the problem persists, contact technical support. 1. Verify that the power supply that contains the fan assembly is fully seated and secured. 2. If the problem persists, replace the power supply. 3. If the problem persists, contact technical support. 42 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

69 Table 22. Power supply console error messages (continued) Error message ses.status.olerror This message occurs when a critical condition is detected in the indicated disk storage shelf oltage sensor. The storage expansion unit might be able to continue operation. ses.status.currenterror Correctie action 1. Verify that the power supply and the AC line are supplying power. 2. Monitor the power grid for abnormalities. 3. Replace the power supply. 4. If the problem persists, contact technical support. This message occurs when a critical condition is detected in the indicated storage shelf current sensor. The storage expansion unit might be able to continue operation. ses.status.temperatureerror This message occurs when the indicated storage expansion unit temperature sensor reports a temperature that exceeds the specifications for the storage expansion unit or its components. 1. Verify that the ambient temperature where the storage expansion unit is installed is within equipment specifications using the enironment shelf [adapter] command, and that airflow clearances are maintained. For clustered systems, you must run this command from the nodeshell of the target node. 2. If the same storage expansion unit also reports fan or fan module failures, correct that problem now. If the problem is reported by the ambient temperature sensor (located on the operator panel), erify that the connection between the storage expansion unit and the panel is secure, if possible. 3. If the problem persists, and if the storage expansion unit has multiple temperature sensors of which only one exhibits the problem, replace the module that contains the sensor that reports the error. If the problem persists, contact technical support for assistance. Note: You can display temperature thresholds for each storage expansion unit through the enironment shelf command. For clustered systems, you must run this command from the nodeshell of the target node. Managing the storage expansion unit 43

70 Table 22. Power supply console error messages (continued) Error message Correctie action ses.psu.powerreqerror This message occurs when too few power supplies are installed to redundantly satisfy the current-draw requirements of the disk dries in the storage expansion unit. This might occur if a power supply is remoed or fails. Some disk drie models require more power than others. If the storage expansion unit specifications for the installed disk drie models specify more power supplies to support that disk type, then this condition can also occur at disk swap or insertion in some storage expansion units. Verify that the number of power supplies installed satisfies the power requirements of the installed disk dries. SAS-Shelf24 requires power supplies in power supply bays 1 and 4 for proper cooling and air flow. If any disk dries are 10K RPM or faster, then power supply bays 2 and 3 must also hae power supplies. ses.psu.coolingreqerror This message occurs when the installed power supplies are placed so that air-flow requirements of the storage expansion unit are not met. The power supply chassis and their power supplies are an integral part of the storage expansion unit cooling and air-flow design. Verify that the power supplies are placed in the locations required to proide proper air flow according to the storage expansion unit specifications. SAS-Shelf24 always requires power supplies in power supply bays 1 and 4 for proper air flow and cooling. Replacing storage expansion unit components This section describes how to replace a storage expansion unit and storage expansion unit components. The following topics are discussed: Replacing SAS cables Replacing SAS cables in a multipath HA or single-controller dual-path (multipath) configuration on page 46 Replacing SAS cables in a single-controller-single-path or single-controllermixed-path configuration on page 48 Hot-swapping a disk drie on page 49 Hot-swapping or replacing an IOM on page 51 Replacing a power supply on page 53 Replacing SAS cables You can replace SAS cables SAS copper and SAS optical cables for shelf-to-shelf and controller-to-shelf connections (using the same SAS ports) if a cable has failed, if a longer cable is needed, if SAS optical cables are preferred instead of SAS copper cables, or if SAS copper cables are necessary instead of SAS optical cables. Depending on your system configuration, this can be done nondisruptiely; otherwise, the procedure requires you to halt your system. Before you begin Note: If you hae a MetroCluster configuration, see the appropriate MetroCluster document for replacing SAS cables. 44 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

71 For stretch MetroCluster configurations using SAS optical cables, see the Configuring a stretch MetroCluster system with SAS disk sheles and SAS optical cables document. For fabric and stretch MetroCluster configurations using FibreBridge 6500N bridges and SAS copper cables, see the Configuring a MetroCluster system with SAS disk sheles and FibreBridge 6500N bridges document. These documents can be found at: If you are using SAS optical cables, your system platform, disk sheles, and ersion of Data ONTAP that your system is running must support SAS optical cables. The most current support information can be found in the Hardware Unierse (formerly the System Configuration Guide) at support/nseries/. If you are replacing one length of SAS copper cable for another, you must erify that the new SAS copper cable is supported by your platform and the ersion of Data ONTAP your system is running. The SAS cables can be SAS copper, SAS optical, or a mix depending on whether or not your system meets the requirements for using the type of cable. If you are using a mix of SAS copper cables and SAS optical cables, the following rules apply: Shelf-to-shelf connections in a stack must be all SAS copper cables or all SAS optical cables. If the shelf-to-shelf connections are SAS optical cables, the shelf-to-controller connections to that stack must also be SAS optical cables. If the shelf-to-shelf connections are SAS copper cables, the shelf-to-controller connections to that stack can be SAS optical cables or SAS copper cables. SAS optical multimode QSFP-to-QSFP cables can be used for controller-to-shelf and shelf-to-shelf connections, and are aailable in lengths up to 50 meters. If you are using SAS optical multimode MPO cables with MPO QSFP modules, the following parameters apply: You can use these cables for controller-to-shelf and shelf-to-shelf connections. The length of a single cable cannot exceed 150 meters for OM4 and 100 meters for OM3. The total end-to-end path (sum of point-to-point paths from the controller to the last shelf) cannot exceed 510 meters. The total path includes the set of breakout cables, patch panels, and inter-panel cables. If you are using SAS optical multimode breakout cables, the following parameters apply: You can use these cables for controller-to-shelf and shelf-to-shelf connections. If you use multimode breakout cables for a shelf-to-shelf connection, you can only use it once within a stack of disk sheles. You must use SAS optical multimode QSFP-to-QSFP or MPO cables with MPO QSFP modules to connect the remaining shelf-to-shelf connections. The point-to-point (QSFP-to-QSFP) path of any multimode cable cannot exceed 150 meters for OM4 and 100 meters for OM3. The path includes the set of breakout cables, patch panels, and inter-panel cables. The total end-to-end path (sum of point-to-point paths from the controller to the last shelf) cannot exceed 510 meters. Managing the storage expansion unit 45

72 The total path includes the set of breakout cables, patch panels, and inter-panel cables. Up to one pair of patch panels can be used in a path. You need to supply the patch panels and inter-panel cables. The inter-panel cables must be the same mode as the SAS optical breakout cable: multimode. You receied a set of QSFP-to-MPO cable modules with each set of SAS optical breakout cables, which you must attach to the MPO end of each SAS optical breakout cable. The breakout cables hae SC, LC, or MTRJ connectors on the opposite end, which connect to a patch panel. You must connect all eight (four pairs) of the SC, LC, or MTRJ breakout connectors to the patch panel. About this task Replacing a SAS cable means that you are replacing a cable using the same exact ports for a controller-to-shelf or shelf-to-shelf connection. Attention: After your storage system is up and sering data, you cannot moe SAS cables (change the SAS ports to which a cable is connected) nondisruptiely. If you need to correct system cabling, you can use a maintenance period to do so. The procedures for replacing SAS cables address multipath HA, single-controller-dual-path (multipath), single-controller-single-path, and single-controller-mixed-path configurations. Disk sheles connected with SAS optical cables require a ersion of disk shelf firmware that supports SAS optical cables. Best practice is to update all disk sheles in the storage system with the latest ersion of disk shelf firmware. Note: Do not reert disk shelf firmware to a ersion that does not support SAS optical cables. You cannot change any disks, disk sheles, or components of a controller module as part of these procedures. Replacing SAS cables in a multipath HA or single-controller dual-path (multipath) configuration You can replace SAS cables nondisruptiely in a multipath HA or single-controller dual-path (multipath) configuration. About this task CAUTION: To perform this procedure, your system must be a multipath HA or single-controller dual path (multipath) configuration. Procedure 1. Verify that your system configuration is Multi-Path HA or Multi-Path (single-controller dual-path) by entering the following command at the system console: sysconfig For HA pairs, you can run the command from either controller. 46 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

73 Note: It might take up to a minute for the system to complete discoery. The configuration is listed in the System Storage configuration field. It should be the fourth line of output.caution: If your system configuration is shown as something other than Multi-Path HA or Multi-Path, you cannot continue with this procedure. 2. Verify that your system meets the requirements listed in the Replacing SAS cables section. 3. If you are replacing SAS copper cables with SAS optical cables, erify that the disk sheles in the storage system hae the latest ersion of disk shelf firmware by completing the following substeps; otherwise, go to Step 5. a. Enter the applicable command: For 7-Mode, enter the following command at the system console: sasadmin expander_map For HA pairs, you can run this command on either node. For clustered systems, enter the following command at the clustershell prompt: run -node node_name -command sasadmin expander_map b. Locate the disk shelf firmware information for the disk sheles in the output is the disk shelf firmware ersion for shelf number one (Slot A/IOM A) in the storage system: Expanders on channel 4a: Leel 3: WWN 500a ff, ID 1, Serial Number ' SHU G114C', Product 'DS424IOM6 ', Re '0151', Slot A c. Compare the firmware information in the command output with the disk shelf firmware information at to determine the most current disk shelf firmware ersion. 4. The next step depends on how current the disk shelf firmware is on your disk sheles. If the firmware ersion in the command output is... The same as or later than the most current ersion on the N series support website (accessed and naigated as described in Websites on page xxi) An earlier ersion than the most current ersion on the N series support website (accessed and naigated as described in Websites on page xxi Then... No disk shelf firmware update is needed. Download the disk shelf firmware file by using the procedure at storage/support/nseries. For HA pairs, you can run the commands from either controller. 5. Replace SAS cables by completing the following substeps: Note: When replacing a SAS cable, wait a minimum of 10 seconds before plugging in the new cable so that the system can detect the cable change. You can ignore cabling messages that might appear on the console. a. Replace cables on side A one cable at a time. The Side A cables are the cables connected to IOM A of each disk shelf. b. Verify that you hae correctly replaced the SAS cables by entering the following command at the console of either controller: sysconfig Managing the storage expansion unit 47

74 The output should be the same as Step 1: the system should be Multi-Path HA, and the SAS port and attached disk shelf information should be the same. If the output is something other than Multi-Path HA, you must identify the cabling error, correct it, and run the sysconfig command again. c. Repeat substeps a and b for Side B. The Side B cables are the cables connected to IOM B of each disk shelf. Replacing SAS cables in a single-controller-single-path or single-controller-mixed-path configuration You must halt your system before replacing SAS cables in a single-controller-singlepath or single-controller-mixed-path configuration. Procedure 1. Verify that your system configuration is Single-Path or Mixed-Path by entering the following command at the system console: sysconfig The configuration is listed in the System Storage configuration field. It should be the fourth line of output. 2. Verify that your system meets the requirements listed in the Replacing SAS cables section. 3. If you are replacing SAS copper cables with SAS optical cables, erify that the disk sheles in the storage system hae the latest ersion of disk shelf firmware by completing the following substeps: otherwise, go to Step 5. a. Enter the applicable command: For 7-Mode, enter the following command at the system console: sasadmin expander_map For HA pairs, you can run this command on either node. For clustered systems, enter the following command at the clustershell prompt: run -node node_name -command sasadmin expander_map b. Locate the disk shelf firmware information for the disk sheles in the output is the disk shelf firmware ersion for shelf number one (Slot A/IOM A) in the storage system: Expanders on channel 4a: Leel 3: WWN 500a ff, ID 1, Serial Number ' SHU G114C', Product 'DS424IOM6 ', Re '0151', Slot A c. Compare the firmware information in the command output with the disk shelf firmware information at to determine the most current disk shelf firmware ersion. 4. The next step depends on how current the disk shelf firmware is on your disk sheles. If the firmware ersion in the command output is... The same as the most current ersion on the N series support website (accessed and naigated as described in Websites on page xxi) An earlier ersion than the most current ersion on the N series support website (accessed and naigated as described in Websites on page xxi) Then... Complete Steps 5 through 7. No disk shelf firmware update is needed. Complete Steps 5 through 8. You need to update the disk shelf firmware. 48 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

75 5. Enter the applicable command from the system console to perform a clean shutdown: For... 7-Mode Clustered systems Then issue this command... halt halt local 6. Replace any SAS cables. If replacing more than one SAS cable, do so one at a time to aoid confusion. 7. Verify that you hae correctly replaced cables by bringing your system back up. a. Boot the system. b. Enter the following command at the system console: sysconfig The output should be the same as Step 1: The system should be Single-Path or Mixed-Path and the SAS port and attached disk-shelf information should be the same. 8. If in Step 4, you had an earlier ersion of disk shelf firmware than the most current ersion on the N series support website (accessed and naigated as described in Websites on page xxi), download the disk shelf firmware file by using the procedure at Hot-swapping a disk drie When a disk drie fails, the storage system logs a warning message to the system console indicating which disk drie in which stack of storage expansion units failed, the fault LED on the operator display panel illuminates, and the fault LED on the failed disk drie illuminates. Replacing a disk drie in a storage expansion unit consists of the following procedures: Remoing a disk drie on page 50 Installing a disk drie on page 50 Note: 1. When replacing seeral disk dries in a storage expansion unit, or when installing seeral disk dries into a half-empty storage unit, replace or install the disk dries one at a time to allow the storage system to recognize the existence of each new disk drie. 2. When installing disk dries in an empty storage expansion unit, power off the storage expansion unit and install all new disk dries. After you hae installed all new disk dries, continue with one of the following tasks, as appropriate: If the storage expansion unit is already cabled to the storage system, power on the storage expansion unit and allow the storage system to recognize the new dries. If the storage expansion unit is not yet cabled to the storage system, refer to Hot-adding a storage expansion unit to an existing system on page 23 or Installing a storage expansion unit for a new storage system on page 15, as appropriate. Managing the storage expansion unit 49

76 300GB 300GB 300GB 300GB Remoing a disk drie About this task To remoe a disk drie, complete the following steps: 1. Ground yourself to the storage system chassis using the grounding leash, and then remoe the front bezel. 2. To remoe the disk drie, press the release button on the far left of the carrier face front. The spring-loaded cam handle releases, opens to its fully opened position, and disengages from the backplane. 300GB 300GB 300GB exn3h082 Figure 8. Releasing a disk drie exn3h083 Figure 9. Remoing a disk drie 3. Use the cam handle to gently slide the disk drie out of the storage expansion unit. Make sure that the disk drie has stopped spinning, which can take 30 seconds. CAUTION: When remoing a disk drie, always use two hands to support its weight. 4. To install a replacement disk drie, go to Installing a disk drie. Installing a disk drie To install a disk drie in a storage expansion unit, complete the following steps: 1. Ground yourself to the storage system chassis using the grounding leash. 2. With the cam handle in the open position, insert the disk drie into the storage expansion unit slot, firmly pushing until the disk drie stops about half an inch from the face of the storage expansion unit. Attention: Do not slam the disk drie into place. 3. Close the cam handle so that it clicks into its locked position. The disk drie fully seats itself. 50 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

77 4. If you are replacing another disk drie, repeat the steps in Remoing a disk drie on page When you are finished replacing or installing all disk dries, replace the front bezel. Hot-swapping or replacing an IOM The IOM connects indiidual disk dries to the rest of the storage system and controls the storage expansion unit operator display panel LEDs. The IOM communicates with the storage system through the SAS port connected to the backplane of the storage expansion unit. The IOM is plugged into the midplane at the back of the storage expansion unit enclosure. For an illustration showing the locations of the IOMs installed, see Figure 2 on page 2. About this procedure Your storage system configuration determines if you can hot-swap the IOM or if you must shut down your system to replace the IOM. This procedure includes instructions for both. Hot-swapping an IOM is only supported on multipath HA configurations, single-path HA configurations, and single-controller dual-path configurations. Single-controller configurations with a single-path connection to the storage expansion units require that you shut down your storage system to replace an IOM. Attention: If you attempt to hot-swap the IOM on a storage expansion unit in a single-controller configuration with a single-path connection to the storage expansion unit, you will lose all access to the disk dries on this storage expansion unit as well as those below it. You could also bring down your entire storage system. Hot-swapping or replacing an IOM About this task To hot-swap or replace an IOM, complete the following steps: 1. Verify that the storage system meets the minimum software requirements to support the storage expansion unit and IOM combination. 2. Ground yourself to the storage system chassis using the grounding leash. 3. The next step depends on what type of storage system configuration you hae. If you hae an actie/actie or high aailability multipath configuration, or a single-controller dual-path configuration, then go to step 4 on page 52. If you hae an actie/actie or high aailability single-path configuration, then take oer the target node by entering the following command from the partner node's console: cf takeoer For clustered systems, enter the following command from the clustershell: storage failoer takeoer -bynode partner HA node If you hae a single-controller single-path configuration, then shut down the storage system by entering the following command at the console: halt Managing the storage expansion unit 51

78 Attention: Always use the halt command to perform a clean shutdown. Verify that the LCD display at the front of the storage system displays the following message: Halted 4. Disconnect the cabling from the IOM that you are remoing. 5. As shown in Figure 10 and Figure 11 on page 53, squeeze the latch on the cam handle until it releases, open the cam handle fully to release the IOM from the midplane. Then, using two hands, pull the IOM out of the storage expansion unit. Note: After you remoe the IOM, wait at least 30 seconds before you install the new IOM so that the drier registers the shelf ID correctly. exn3h084 Figure 10. Releasing an IOM 52 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

79 exn3h085 Figure 11. Remoing an IOM 6. With the cam handle in the open position, insert the new IOM into the storage expansion unit, firmly pushing until the IOM meets the midplane, and then close the cam handle so that the latch clicks into the locked position and the IOM is fully seated. Attention: Do not use excessie force when sliding the IOM into the storage expansion unit; you might damage the connector. 7. Reconnect the stack cabling. 8. The next step depends on what type of storage system configuration you hae. If you hae an actie/actie or high aailability multipath configuration, or a single-controller dual-path configuration, then go to step 9. If you hae an actie/actie or high aailability single-path configuration, gie back the target mode by entering the applicable command: cf gieback For clustered systems enter the following command from the clustershell: storage failoer gieback -fromnode partner HA node If you hae a single-controller single-path configuration, then reboot your storage system. 9. Verify that the IOM link has been established by checking that the LNK LEDs in the IOM are illuminated. Replacing a power supply Replacing a power supply in a storage expansion unit consists of the following procedures: Remoing a power supply on page 54 Installing a power supply on page 55 Managing the storage expansion unit 53

80 Attention: When replacing the power supply in your storage expansion unit, obsere the following rules: You do not need to turn off the power when you replace one power supply. If you are replacing more than one power supply in the same storage expansion unit, replace them one at a time to aoid powering down the storage expansion unit. You must replace the power supply within two minutes of remoal to minimize disruption to the storage expansion unit's airflow. The power supplies are plugged into the backplane at the back of the storage expansion unit enclosure. Note: Depending on the types of disk dries, your storage expansion unit requires two or four power supplies. For illustrations showing storage expansion units with two and four power supplies installed, see Figure 2 on page 2 and Figure 3 on page 2. Remoing a power supply About this task To remoe a power supply, complete the following steps: 1. Ground yourself to the storage system chassis using the grounding leash. 2. Turn off the power switch on the power supply that you are replacing. 3. Unplug the power cord from the power source. Then open the power cord retainer and unplug the power cord from the power supply. 4. As shown in Figure 12 and Figure 13 on page 55, squeeze the cam handle latch to release the cam handle, and then open the cam handle to its fully open position. exn3h086 Figure 12. Releasing a power supply 54 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

81 exn3h087 Figure 13. Remoing a power supply 5. Use the cam handle to slide the power supply out of the storage expansion unit. Note: a. When remoing a power supply, always use two hands to support its weight. b. Replace the power supply within two minutes of remoal to minimize disruption to the system airflow. Installing a power supply About this task To install a power supply in a storage expansion unit, complete the following steps. 1. Ground yourself to the storage system chassis using the grounding leash. 2. With the cam handle in the open position, insert the power supply into the storage expansion unit, firmly pushing until the power supply engages the backplane. Attention: Do not use excessie force when sliding the power supply into the storage expansion unit; you can damage the connector. 3. Close the cam handle and with some force, click the cam handle latch into the closed position, which fully seats and secures the power supply in place. 4. Reconnect the power cord to the power supply and then to the power source. 5. Secure the power cord to the power supply using the power cord retainer. 6. Turn on the power switch. Managing the storage expansion unit 55

82 56 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

83 Recommended Power Line Sizes This appendix describes the recommended AC power line lengths running from the N series storage system to the power source. Recommended AC power line sizes By properly designing longer AC power feeds, you can presere oltage leels to the equipment. The wiring from the breaker panel to the power strip, which supplies power to the storage system and storage expansion units, can often exceed 50 feet. Note: Total AC wire length = breaker to wall or ceiling outlet + extension cable or ceiling drop. The following table lists the recommended conductor size for 2% oltage drop for a particular distance in feet (taken from the Radio Engineer's Handbook). Table 23. Recommended conductor size for 2% oltage drop 110V, single-phase 20A circuit 30A circuit 40A circuit 50A circuit 25 feet 12 AWG 10 AWG 8 AWG 8 AWG 50 feet 8 AWG 6 AWG 6 AWG 4 AWG 75 feet 6 AWG 4 AWG 4 AWG 2 AWG 220V, single-phase 20A circuit 30A circuit 40A circuit 50A circuit 25 feet 14 AWG 12 AWG 12 AWG 10 AWG 50 feet 12 AWG 10 AWG 8 AWG 8 AWG 75 feet 10 AWG 8 AWG 6 AWG 6 AWG The following table lists the approximate equialent wire gauge (American Wire Gauge (AWG) to Harmonized Cordage). Table 24. American Wire Gauge (AWG) to Harmonized Cordage AWG Harmonized, mm-mm mm-mm = millimeter squared Copyright IBM Corp. 2009,

84 58 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

85 FRU/CRU and power cord list for N series products This appendix contains information about field-replaceable units (FRUs), customer-replaceable units (CRUs), and power cords for N series products. FRU/CRU list for N series products For the most current FRU/CRU list for your N series product, access the IBM N series support website, as described in Websites on page xxi, and refer to the FRU (Field Replaceable Units) lists. Power cord list for N series products The following list details the power cord feature codes (FCs) for N series products. FC 9000 (All countries) Power cord, Rack PDU 27 inches Rated 250 V/15 A Product end uses C14; PDU end uses C13. FC 9001 Europe and others Proides power cords for Austria, Belgium, Boliia, Bulgaria, Chile, Croatia, Czech Republic, Egypt, Estonia, European Union, Finland, France, Germany, Greece, Hungary, Iceland, Indonesia, Latia, Lebanon, Lithuania, Luxemburg, Morocco, Netherlands, Norway, Peru, Poland, Portugal, Romania, Russia, Sloakia, Sloenia, Spain, Suriname, Sweden, Turkey 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 211 (CEE 7-VII) designed for V ac input. FC 9002 United Kingdom and others Proides power cords for United Kingdom, Costa Rica, Cyprus, Guyana, Hong Kong, Ireland, Kuwait, Malta, Oman, Singapore, Sri Lanka 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 210 (13A fuse) designed for V ac input. FC 9003 Japan Proides power cords for Japan 1.83 m (6 feet), unshielded, rated 125 V/15 A. Attached plug EL 302 (JIS C3306) designed for V ac input. FC 9004 U.S., Six Feet (2 m) Proides power cords for U.S., Canada, Mexico, Belize, Columbia, Ecuador, El Salador, Guatemala, Honduras, Korea, Nicaragua, Panama, Philippines, Puerto Rico, Saudi Arabia, Thailand, Venezuela 1.83 m (6 feet), unshielded, rated 125 V/15 A. Attached plug EL 302 (Nema 5-15P) designed for V ac input. FC 9005 Australia, New Zealand Proides power cords for Australia, New Zealand, Uruguay 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 206 (AS 3112) designed for V ac input. Copyright IBM Corp. 2009,

86 FC 9006 Switzerland, Liechtenstein Proides power cords for Switzerland, Liechtenstein 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 203 (SEV 1011) designed for V ac input. FC 9007 Argentina Proides power cords for Argentina 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 219 (IRAM 2073) designed for V ac input. FC 9008 China Proides power cords for China 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 602 (GB 2099/GB 1002) designed for V ac input. FC 9009 Denmark Proides power cords for Denmark 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 213 (DHCR D1) designed for V ac input. FC 9010 India, Pakistan, South Africa Proides power cords for India, Macau, Pakistan, South Africa 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 208 (BS 164-1, BS 546) designed for V ac input. FC 9011 Israel Proides power cords for Israel 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 212 (SI 32) designed for V ac input. FC 9012 Italy Proides power cords for Italy 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 502 (CEI 23-16) designed for V ac input. FC 9013 North America (250 V) Proides power cords for U.S m (6 feet), unshielded, rated 250 V/15 A. Attached plug EL 309 (NEMA 6-15P) designed for V ac input. FC 9014 Brazil Proides power cords for Brazil 2.5 m (9 feet), unshielded, rated 250 V/10 A. Attached plug EL 211 (NBR 6147/2000) designed for V ac input FC 9015 Taiwan Proides 125 V power cords for Taiwan 2.5 m (9 feet), unshielded, rated 125 V/15 A. Attached plug EL 302 (CNS ) designed for V ac input. FC 9016 Taiwan (250 V) Proides 250 V power cords for Taiwan 1.83 m (6 feet), unshielded, rated 250 V/10 A. 60 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

87 Attached plug EL 610 (CNS 10917, CNS 690) designed for 250 V ac input. FRU/CRU and power cord list for N series products 61

88 62 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

89 Notices This information was deeloped for products and serices offered in the U.S.A. IBM may not offer the products, serices, or features discussed in this document in other countries. Consult your local IBM representatie for information on the products and serices currently aailable in your area. Any reference to an IBM product, program, or serice is not intended to state or imply that only that IBM product, program, or serice may be used. Any functionally equialent product, program, or serice that does not infringe on any IBM intellectual property right may be used instead. Howeer, it is the user's responsibility to ealuate and erify the operation of any non-ibm product, program, or serice. IBM may hae patents or pending patent applications coering subject matter described in this document. The furnishing of this document does not gie you any license to these patents. You can send license inquiries, in writing to: IBM Director of Licensing IBM Corporation North Castle Drie Armonk, N.Y U.S.A. For additional information, isit the web at: contact/ The following paragraph does not apply to the United Kingdom or any other country where such proisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-ibm websites are proided for conenience only and do not in any manner sere as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. IBM may use or distribute any of the information you supply in any way it beliees appropriate without incurring any obligation to you. Any performance data contained herein was determined in a controlled enironment. Therefore, the results obtained in other operating enironments may ary significantly. Some measurements may hae been made on deelopment-leel Copyright IBM Corp. 2009,

90 systems and there is no guarantee that these measurements will be the same on generally aailable systems. Furthermore, some measurement may hae been estimated through extrapolation. Actual results may ary. Users of this document should erify the applicable data for their specific enironment. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly aailable sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. If you are iewing this information in softcopy, the photographs and color illustrations may not appear. Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. A complete and current list of other IBM trademarks is aailable on the web at Important notes NetApp, the NetApp logo, Network Appliance, the Network Appliance logo, Akorri, ApplianceWatch, ASUP, AutoSupport, BalancePoint, BalancePoint Predictor, Bycast, Campaign Express, ComplianceClock, Cryptainer, CryptoShred, CyberSnap, Data Center Fitness, Data ONTAP, DataFabric, DataFort, Decru, Decru DataFort, DenseStak, Engenio, Engenio logo, E-Stack, ExpressPod, FASerer, FastStak, FilerView, Flash Accel, Flash Cache, Flash Pool, FlashRay, FlexCache, FlexClone, FlexPod, FlexScale, FlexShare, FlexSuite, FlexVol, FPolicy, GetSuccessful, gfiler, Go further, faster, Imagine Virtually Anything, Lifetime Key Management, LockVault, Mars, Manage ONTAP, MetroCluster, MultiStore, NearStore, NetCache, NOW (NetApp on the Web), Onaro, OnCommand, ONTAPI, OpenKey, PerformanceStak, RAID-DP, ReplicatorX, SANscreen, SANshare, SANtricity, SecureAdmin, SecureShare, Select, Serice Builder, Shadow Tape, Simplicity, Simulate ONTAP, SnapCopy, Snap Creator, SnapDirector, SnapDrie, SnapFilter, SnapIntegrator, SnapLock, SnapManager, SnapMigrator, SnapMirror, SnapMoer, SnapProtect, SnapRestore, Snapshot, SnapSuite, SnapValidator, SnapVault, StorageGRID, StoreVault, the StoreVault logo, SyncMirror, Tech OnTap, The eolution of storage, Topio, VelocityStak, Filer, VFM, Virtual File Manager, VPolicy, WAFL, Web Filer, and XBB are trademarks or registered trademarks of NetApp, Inc. in the United States, other countries, or both. Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Other company, product, or serice names may be trademarks or serice marks of others. Processor speeds indicate the internal clock speed of the microprocessor; other factors also affect application performance. 64 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

91 Electronic emission notices CD-ROM drie speeds list the ariable read rate. Actual speeds ary and are often less than the maximum possible. When referring to processor storage, real and irtual storage, or channel olume, KB stands for approximately 1000 bytes, MB stands for approximately bytes, and GB stands for approximately bytes. When referring to hard disk drie capacity or communications olume, MB stands for bytes, and GB stands for bytes. Total user-accessible capacity may ary depending on operating enironments. Maximum internal hard disk drie capacities assume the replacement of any standard hard disk dries and population of all hard disk drie bays with the largest currently supported dries aailable from IBM. Maximum memory may require replacement of the standard memory with an optional memory module. IBM makes no representation or warranties regarding non-ibm products and serices that are SererProen, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. These products are offered and warranted solely by third parties. IBM makes no representations or warranties with respect to non-ibm products. Support (if any) for the non-ibm products is proided by the third party, not IBM. Some software may differ from its retail ersion (if aailable), and may not include user manuals or all program functionality. Attention: In compliance with the GNU General Public License (GPL), Version 2, June 1991, a complete machine-readable copy of the source code for the releant source code portions of the Remote LAN Module (RLM) Firmware that are coered by the GPL, is aailable from ftp://ftp.netapp.com/frm-ntap/opensource/. This section contains the electronic emission notices or statements for the United States and other countries. Federal Communications Commission Statement This explains the Federal Communications Commission's (FCC s) statement. This equipment has been tested and found to comply with the limits for a Class A digital deice, pursuant to Part 15 of the FCC Rules. These limits are designed to proide reasonable protection against harmful interference when the equipment is operated in a commercial enironment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, might cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case the user will be required to correct the interference at his own expense. Properly shielded and grounded cables and connectors must be used in order to meet FCC emission limits. IBM is not responsible for any radio or teleision interference caused by using other than recommended cables and connectors, or by Notices 65

92 unauthorized changes or modifications to this equipment. Unauthorized changes or modifications could oid the user's authority to operate the equipment. This deice complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) this deice might not cause harmful interference, and (2) this deice must accept any interference receied, including interference that might cause undesired operation. Industry Canada Compliance Statement This Class A digital apparatus complies with Canadian ICES-003. Cet appareil numérique de la classe A est conform à la norme NMB-003 du Canada. Australia and New Zealand Class A Statement Attention: This is a Class A product. In a domestic enironment this product might cause radio interference in which case the user might be required to take adequate measures. European Union Electromagnetic Compatibility Directie This product is in conformity with the protection requirements of European Union (EU) Council Directie 2004/108/EC on the approximation of the laws of the Member States relating to electromagnetic compatibility. IBM cannot accept responsibility for any failure to satisfy the protection requirements resulting from a non-recommended modification of the product, including the fitting of non-ibm option cards. Attention: This is an EN Class A product. In a domestic enironment this product might cause radio interference in which case the user might be required to take adequate measures. Responsible Manufacturer: International Business Machines Corp. New Orchard Road Armonk, New York European community contact: IBM Deutschland GmbH Technical Regulations, Department M372 IBM-Allee 1, Ehningen, Germany Tele: lugi@de.ibm.com Germany Electromagnetic Compatibility Directie Deutschsprachiger EU Hinweis: Hinweis für Geräte der Klasse A EU-Richtlinie zur Elektromagnetischen Verträglichkeit Dieses Produkt entspricht den Schutzanforderungen der EU-Richtlinie 2004/108/EG zur Angleichung der Rechtsorschriften über die elektromagnetische Verträglichkeit in den EU-Mitgliedsstaaten und hält die Grenzwerte der EN Klasse A ein. 66 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

93 Um dieses sicherzustellen, sind die Geräte wie in den Handbüchern beschrieben zu installieren und zu betreiben. Des Weiteren dürfen auch nur on der IBM empfohlene Kabel angeschlossen werden. IBM übernimmt keine Verantwortung für die Einhaltung der Schutzanforderungen, wenn das Produkt ohne Zustimmung der IBM erändert bzw. wenn Erweiterungskomponenten on Fremdherstellern ohne Empfehlung der IBM gesteckt/eingebaut werden. EN Klasse A Geräte müssen mit folgendem Warnhinweis ersehen werden: "Warnung: Dieses ist eine Einrichtung der Klasse A. Diese Einrichtung kann im Wohnbereich Funk-Störungen erursachen; in diesem Fall kann om Betreiber erlangt werden, angemessene Mabnahmen zu ergreifen und dafür aufzukommen." Deutschland: Einhaltung des Gesetzes über die elektromagnetische Verträglichkeit on Geräten Dieses Produkt entspricht dem "Gesetz über die elektromagnetische Verträglichkeit on Geräten (EMVG)." Dies ist die Umsetzung der EU-Richtlinie 2004/108/EG in der Bundesrepublik Deutschland. Zulassungsbescheinigung laut dem Deutschen Gesetz über die elektromagnetische Verträglichkeit on Geräten (EMVG) (bzw. der EMC EG Richtlinie 2004/108/EG) für Geräte der Klasse A Dieses Gerät ist berechtigt, in übereinstimmung mit dem Deutschen EMVG das EG-Konformitätszeichen - CE - zu führen. Verantwortlich für die Einhaltung der EMV Vorschriften ist der Hersteller: International Business Machines Corp. New Orchard Road Armonk,New York Tel: Der erantwortliche Ansprechpartner des Herstellers in der EU ist: IBM Deutschland GmbH Technical Regulations, Abteilung M372 IBM-Allee 1, Ehningen, Germany Tele: lugi@de.ibm.com Generelle Informationen: Das Gerät erfüllt die Schutzanforderungen nach EN und EN Klasse A. Notices 67

94 People's Republic of China Class A Statement Taiwan Class A Statement Taiwan Contact Information This topic contains the product serice contact information for Taiwan. IBM Taiwan Product Serice Contact Information: IBM Taiwan Corporation 3F, No 7, Song Ren Rd., Taipei Taiwan Tel: f2c00790 Japan Voluntary Control Council for Interference Class A Statement This explains the Japan Voluntary Control Council for Interference (VCCI) statement. 68 IBM System Storage EXN3000 Storage Expansion Unit: Hardware and Serice Guide

95 Japan Electronics and Information Technology Industries Association Statement This explains the Japan Electronics and Information Technology Industries Association (JEITA) statement for less than or equal to 20 A per phase. This explains the JEITA statement for greater than 20 A per phase. Korean Communications Commission Class A Statement This explains the Korean Communications Commission (KCC) statement. Russia Electromagnetic Interference Class A Statement This statement explains the Russia Electromagnetic Interference (EMI) statement. rusemi jjieta2 jjieta1 Power cords For your safety, IBM proides a power cord with a grounded attachment plug to use with this IBM product. To aoid electrical shock, always use the power cord and plug with a properly grounded outlet. IBM power cords used in the United States and Canada are listed by Underwriter's Laboratories (UL) and certified by the Canadian Standards Association (CSA). Notices 69