N6000 Series Hardware and Service Guide

Transcription

1 IBM System Storage N6000 Series Hardware and Service Guide Covering the N6040, N6060 and N6070 models GC

2

3 IBM System Storage N6000 Series Hardware and Service Guide Covering the N6040, N6060 and N6070 models GC

4 Note: Before using this information and the product it supports, be sure to read the general information in Notices on page 113. The following paragraph does not apply to any country (or region) where such provisions are inconsistent with local law. INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states (or regions) do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement may not apply to you. Order publications through your IBM representative or the IBM branch office serving your locality. Copyright IBM Corporation 2008, US Government Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

5 Safety notices Safety notices and labels This section contains information about: v Safety notices and labels v Laser safety on page vi v Rack safety on page vii v Fire suppression systems on page ix For information about environmental notices, see the IBM Environmental Notices and User Guide. When using this product, observe the danger, caution, and attention notices contained in this guide. The notices are accompanied by symbols that represent the severity of the safety condition. The following sections define each type of safety notice and provide examples. The following notices and statements are used in IBM documents. They are listed below in order of increasing severity of potential hazards. Follow the links for more detailed descriptions and examples of the danger, caution, and attention notices in the sections that follow. v Note: These notices provide important tips, guidance, or advice. v v v v Attention notices on page v: These notices indicate potential damage to programs, devices, or data. Caution notices on page v: These statements indicate situations that can be potentially hazardous to you. Danger notices : These statements indicate situations that can be potentially lethal or extremely hazardous to you. Safety labels are also attached directly to products to warn of these situations. In addition to these notices, Labels on page v may be attached to the product to warn of potential hazards. Danger notices A danger notice calls attention to a situation that is potentially lethal or extremely hazardous to people. A lightning bolt symbol accompanies a danger notice to represent a dangerous electrical condition. A sample danger notice follows. DANGER An electrical outlet that is not correctly wired could place hazardous voltage on metal parts of the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock. (D004) Copyright IBM Corp. 2008, 2013 iii

6 A general danger notice provides instructions on how to avoid shock hazards when servicing equipment. Unless instructed otherwise, follow the procedures in the following danger notice. DANGER When working on or around the system, observe the following precautions: Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard: v Connect power to this unit only with the IBM provided power cord. Do not use the IBM provided power cord for any other product. v Do not open or service any power supply assembly. v Do not connect or disconnect any cables or perform installation, maintenance, or reconfiguration of this product during an electrical storm. v The product might be equipped with multiple power cords. To remove all hazardous voltages, disconnect all power cords. v Connect all power cords to a properly wired and grounded electrical outlet. Ensure outlet supplies proper voltage and phase rotation according to the system rating plate. v Connect any equipment that will be attached to this product to properly wired outlets. v When possible, use one hand only to connect or disconnect signal cables. v Never turn on any equipment when there is evidence of fire, water, or structural damage. v Disconnect the attached power cords, telecommunications systems, networks, and modems before you open the device covers, unless instructed otherwise in the installation and configuration procedures. v Connect and disconnect cables as described below when installing, moving, or opening covers on this product or attached devices. To Disconnect: 1. Turn off everything (unless instructed otherwise). 2. Remove power cords from the outlet. 3. Remove signal cables from connectors. 4. Remove all cables from devices. To Connect: 1. Turn off everything (unless instructed otherwise). 2. Attach all cables to devices. 3. Attach signal cables to the connectors. 4. Attach power cords to the outlets. 5. Turn on the devices. (D005) iv IBM System Storage: N6000 Series Hardware and Service Guide

7 Labels As an added precaution, safety labels are often installed directly on products or product components to warn of potential hazards. The actual product safety labels may differ from these sample safety labels: DANGER Hazardous voltage, current, or energy levels are present inside any component that has this label attached. Do not open any cover or barrier that contains this label. (L001) (L003) DANGER Multiple power cords. The product might be equipped with multiple power cords. To remove all hazardous voltages, disconnect all power cords. Caution notices A caution notice calls attention to a situation that is potentially hazardous to people because of some existing condition. A caution notice can be accompanied by different symbols, as in the examples below: If the symbol is... It means... A hazardous electrical condition with less severity than electrical danger. A generally hazardous condition not represented by other safety symbols. A hazardous condition due to the use of a laser in the product. Laser symbols are always accompanied by the classification of the laser as defined by the U. S. Department of Health and Human Services (for example, Class I, Class II, and so forth). Attention notices An attention notice indicates the possibility of damage to a program, device, or system, or to data. An exclamation point symbol may accompany an attention notice, but is not required. A sample attention notice follows: Attention: Do not bend a fiber cable to a radius less than 5 cm (2 in.); you can damage the cable. Tie wraps are not recommended for optical cables because they can be easily overtightened, causing damage to the cable. Safety notices v

8 Laser safety CAUTION: This product contains a Class 1M laser. Do not view directly with optical instruments. (C028) This equipment contains Class 1 laser products, and complies with FDA radiation Performance Standards, 21 CFR Subchapter J and the international laser safety standard IEC CAUTION: Data processing environments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical fiber cable or open receptacle. (C027) Attention: In the United States, use only SFP or GBIC optical transceivers that comply with the FDA radiation performance standards, 21 CFR Subchapter J. Internationally, use only SFP or GBIC optical transceivers that comply with IEC standard Optical products that do not comply with these standards may produce light that is hazardous to the eyes. Usage restrictions The optical ports of the modules must be terminated with an optical connector or with a dust plug. vi IBM System Storage: N6000 Series Hardware and Service Guide

9 Rack safety Rack installation DANGER Observe the following precautions when working on or around your IT rack system: v Heavy equipment - personal injury or equipment damage might result if mishandled. v Always lower the leveling pads on the rack cabinet. v Always install stabilizer brackets on the rack cabinet. v To avoid hazardous conditions due to uneven mechanical loading, always install the heaviest devices in the bottom of the rack cabinet. Always install servers and optional devices starting from the bottom of the rack cabinet. v Rack-mounted devices are not to be used as shelves or work spaces. Do not place objects on top of rack-mounted devices. v v v Each rack cabinet might have more than one power cord. Be sure to disconnect all power cords in the rack cabinet when directed to disconnect power during servicing. Connect all devices installed in a rack cabinet to power devices installed in the same rack cabinet. Do not plug a power cord from a device installed in one rack cabinet into a power device installed in a different rack cabinet. An electrical outlet that is not correctly wired could place hazardous voltage on the metal parts of the system or the devices that attach to the system. It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock. (R001 part 1 of 2 Safety notices vii

10 CAUTION: v Do not install a unit in a rack where the internal rack ambient temperatures will exceed the manufacturer's recommended ambient temperature for all your rack-mounted devices. v Do not install a unit in a rack where the air flow is compromised. Ensure that air flow is not blocked or reduced on any side, front, or back of a unit used for air flow through the unit. v Consideration should be given to the connection of the equipment to the supply circuit so that overloading of the circuits does not compromise the supply wiring or overcurrent protection. To provide the correct power connection to a rack, refer to the rating labels located on the equipment in the rack to determine the total power requirement of the supply circuit. v (For sliding drawers.) Do not pull out or install any drawer or feature if the rack stabilizer brackets are not attached to the rack. Do not pull out more than one drawer at a time. The rack might become unstable if you pull out more than one drawer at a time. v (For fixed drawers) This drawer is a fixed drawer and should not be moved for servicing unless specified by manufacturer. Attempting to move the drawer partially or completely out of the rack may cause the rack to become unstable or cause the drawer to fall out of the rack. (R001 part 2 of 2) viii IBM System Storage: N6000 Series Hardware and Service Guide

11 Rack relocation (19" rack) Fire suppression systems CAUTION: Removing components from the upper positions in the rack cabinet improves rack stability during relocation. Follow these general guidelines whenever you relocate a populated rack cabinet within a room or building: v Reduce the weight of the rack cabinet by removing equipment starting at the top of the rack cabinet. When possible, restore the rack cabinet to the configuration of the rack cabinet as you received it. If this configuration is not known, you must do the following: Remove all devices in the 32U position and above. Ensure that the heaviest devices are installed in the bottom of the rack cabinet. Ensure that there are no empty U-levels between devices installed in the rack cabinet below the 32U level. If the rack cabinet you are relocating is part of a suite of rack cabinets, detach the rack cabinet from the suite. Inspect the route that you plan to take when moving the rack to eliminate potential hazards. Verify that the route that you choose can support the weight of the loaded rack cabinet. Refer to the documentation that came with your rack cabinet for the weight of a loaded rack cabinet. Verify that all door openings are at least 760 x 2030 mm (30 x 80 in.). Ensure that all devices, shelves, drawers, doors, and cables are secure. Ensure that the four leveling pads are raised to their highest position. Ensure that there is no stabilizer bracket installed on the rack cabinet during movement. Do not use a ramp inclined at more than ten degrees. Once the rack cabinet is in the new location, do the following: - Lower the four leveling pads. - Install stabilizer brackets on the rack cabinet. - If you removed any devices from the rack cabinet, repopulate the rack cabinet from the lowest position to the highest position. If a long distance relocation is required, restore the rack cabinet to the configuration of the rack cabinet as you received it. Pack the rack cabinet in the original packaging material, or equivalent. Also, lower the leveling pads to raise the casters off of the pallet and bolt the rack cabinet to the pallet. (R002) A fire suppression system is the responsibility of the customer. The customer's own insurance underwriter, local fire marshal, or a local building inspector, or both, should be consulted in selecting a fire suppression system that provides the correct level of coverage and protection. IBM designs and manufactures equipment to internal and external standards that require certain environments for reliable operation. Because IBM does not test any equipment for compatibility with fire suppression systems, IBM does not make compatibility claims of any kind nor does IBM provide recommendations on fire suppression systems. Safety notices ix

12 x IBM System Storage: N6000 Series Hardware and Service Guide

13 Contents Safety notices iii Safety notices and labels iii Danger notices iii Labels v Caution notices v Attention notices v Laser safety vi Usage restrictions vi Rack safety vii Rack installation vii Rack relocation (19" rack) ix Fire suppression systems ix Figures xv Tables xvii About this document xix Who should read this document xix Related documents xix Supported features xx Websites xx Getting information, help, and service xx Before you call xxi Using the documentation xxi Hardware service and support xxi Firmware updates xxi Terminology and conventions used in this document xxii Active/active and high-availability terms xxii Storage terms xxii Command conventions xxiii Formatting conventions xxiii Keyboard conventions xxiv How to send your comments xxiv Chapter 1. Preparing for the installation Required manuals, tools and equipment Handling static-sensitive devices Planning and organizing the installation Telecommunication regulatory statement Hardware specifications Checking shipment package contents Rules for installing the system in a rack Guide to the installation process Chapter 2. Connecting an N6000 series system Handling fiber-optic cables Connecting your system to an IP network About the e0m interface Connecting your system to a power source Connecting your system to storage Connecting a filer to storage expansion units Connecting a gateway to external storage Connecting to a third-party device Copyright IBM Corp. 2008, 2013 xi

14 Connecting your system to an ASCII terminal console Chapter 3. Configuring an N6000 series system Configuring the N6000 series system Configuring the Fibre Channel port N6000 series system active/active or high availability configurations Configuring for initiator mode Configuring and using the RLM Chapter 4. Replacing N6000 series system devices Using the IBM N series support website Replacing a CompactFlash card in N6000 series systems running Data ONTAP 7.x Verifying system requirements Replacing a CompactFlash card in a single-controller system running Data ONTAP 7.x Replacing a CompactFlash card in an active/active configuration running Data ONTAP 7.x Replacing a CompactFlash card in a system running in Data ONTAP 8.0 (7-Mode) Shutting down a node operating in 7-Mode Opening the system Removing the CompactFlash card from the controller module Installing the CompactFlash card Completing the replacement process Replacing a CompactFlash card in a system running Data ONTAP 8.2 (Cluster Mode) Shutting down a node operating in clustered Data ONTAP Opening the system Removing the CompactFlash card from the controller module Installing the CompactFlash card Completing the replacement process Replacing a fan module in a system Removing a fan module Installing a fan module Completing the replacement process Installing or replacing the RLM in a system Minimum required RLM firmware version Shutting down the node Opening the system Removing the RLM Installing the RLM Reinstalling the controller module and booting the system Connecting the RLM Setting up a new RLM Verifying the firmware version and testing an RLM Running diagnostics on the RLM Completing the replacement process Replacing a power supply on a system Removing a power supply Installing a power supply Completing the replacement process Replacing the Real-Time Clock (RTC) battery in a system Shutting down the node Opening the system Removing an RTC battery Installing an RTC battery Reinstalling the controller module and booting the system Running diagnostics on the RTC battery Resetting the date and time on the system Completing the replacement process Replacing an NVRAM battery in a system Shutting down the node Opening the system Removing an NVRAM battery xii IBM System Storage: N6000 Series Hardware and Service Guide

15 Installing an NVRAM battery Reinstalling the controller module and booting the system Checking the status of the NVRAM battery Completing the replacement process Replacing a DIMM in a system Shutting down the node Opening the system Removing the DIMMs Installing the DIMMs Reinstalling the controller module and booting the system Running diagnostics on the DIMM Completing the replacement process Replacing PCI cards and risers in a system Shutting down the node Opening the system Removing a PCIe card Installing a PCIe card Removing a riser Installing a riser Reinstalling the controller module and booting the system Running diagnostics on a PCIe card Configuring newly added Ethernet ports for cluster or management use (Data ONTAP 8.2 Cluster-Mode) Completing the replacement process Appendix A. Recommended power line sizes Recommended AC power line sizes Appendix B. FRU/CRU and power cord list for N series products FRU/CRU list for N series products Power cord list for N series products Notices Trademarks Important notes Electronic emission notices Federal Communications Commission (FCC) Class A Statement Industry Canada Class A Emission Compliance Statement Avis de conformité à la réglementation d'industrie Canada European Union EMC Directive Conformance Statement Germany Electromagnetic Compatibility Directive People's Republic of China Class A Electronic Emission Statement Japan Voluntary Control Council for Interference (VCCI) Class A Statement Japan Electronics and Information Technology Industries Association (JEITA) Statement Korea Communications Commission (KCC) Class A Statement Russia Electromagnetic Interference (EMI) Class A Statement Taiwan Class A Electronic Emission Statement Australia and New Zealand Class A Statement Power cords Index Contents xiii

16 xiv IBM System Storage: N6000 Series Hardware and Service Guide

17 Figures 1. Onboard port locations (Ethernet) e0m interface Onboard port locations (Fibre Channel) Cabling ports 0b and 0d to switches Cabling ports 0a and 0c to storage expansion units Cabling expansion cards to storage expansion units for dual-path Fibre Channel cabling N6000 series system components - Front N6000 series system components - Rear N6000 series exterior components N6000 series system components - Exploded view CompactFlash card removal Replacing a fan module Replacing a power supply RTC battery removal NVRAM battery location Replacing DIMMs Lifting the DIMM from the slot Replacing a PCIe card and riser Copyright IBM Corp. 2008, 2013 xv

18 xvi IBM System Storage: N6000 Series Hardware and Service Guide

19 Tables 1. N6000 series physical characteristics N6000 series clearance dimensions N6000 series environmental requirements N6000 series systems maximum electrical power N6040 electrical requirements one controller module N6040 electrical requirements two controller modules N6040 electrical requirements with two PAM or Flash Cache module cards one controller module N6040 electrical requirements with four PAM or Flash Cache module cards two controller modules N6060 electrical requirements one controller module N6060 electrical requirements two controller modules N6060 electrical requirements with two PAM or Flash Cache module cards one controller module N6060 electrical requirements with four PAM or Flash Cache module cards two controller modules N6070 electrical requirements one controller module N6070 electrical requirements two controller modules N6070 electrical requirements with four PAM or Flash Cache module cards one controller module N6070 electrical requirements with eight PAM or Flash Cache module cards two controller modules Filer installation process procedures RLM configuration manuals V, single phase recommended conductor sizes V, single phase recommended conductor sizes American Wire Gage to Harmonized Cordage equivalents Copyright IBM Corp. 2008, 2013 xvii

20 xviii IBM System Storage: N6000 Series Hardware and Service Guide

21 About this document This guide describes how to install, connect, manage, and service the following IBM System Storage N series systems: v N6040 (model numbers 2858-A10 and 2858-A20) v N6060 (model numbers 2858-A12 and 2858-A22) v N6070 (model numbers 2858-A11 and 2858-A21) Note: Throughout this document, the N6040, N6060, and N6070 systems are referred to generically as N6000 series systems unless information applies only to a specific product, in which case the specific machine name is used. For the latest version of this document and all IBM System Storage N series documentation, go to the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Compliance ID 2858-NAS covers MT/models 2858-A10, 2858-A11, 2858-A20, 2858-A21, 2858-A12, and 2858-A22. Note: This guide applies to systems, including systems with gateway functionality, running Data ONTAP 7.x and Data ONTAP 8.x 7-Mode. In the Data ONTAP 8.x 7-Mode product name, the term 7-Mode signifies that the 8.x release has the same features and functionality found in the prior Data ONTAP 7.1, 7.2, and 7.3 release families. Who should read this document Related documents This guide is for qualified system administrators and service personnel who are familiar with IBM storage systems. This document is for customer use. It addresses setup, operation, and servicing of the 2858 models A10, A11, A20, A21, A12, and A22. This document is intended to provide information to customers, operators, administrators, installers, and service personnel. The following documents, as well as documentation for Data ONTAP and other software, are available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. For information about installation and setup of your N6000 series system, see the Installation and Setup Instructions that came with your system. For information about error messages, troubleshooting, and monitoring the LEDs for your N6000 series system and optional adapter cards, see the IBM System Storage N series Platform Monitoring Guide. For diagnostic information about your N6000 series system, see the IBM System Storage N series Diagnostics Guide. For a list of N series hardware and hardware-related documents, as well as lists of Data ONTAP documentation by release family, refer to the "Bibliography" appendix in the IBM System Storage N series Introduction and Planning Guide. Copyright IBM Corp. 2008, 2013 xix

22 Supported features Websites For a list of optional adapter cards supported by your N6000 series system, refer to the appropriate "Optional adapter cards" appendix in the IBM System Storage N series Introduction and Planning Guide. If you are replacing or installing a Performance Acceleration Module (PAM) or Flash Cache module, refer to the "Installing or replacing a Flash Cache module or PAM" appendix in the IBM System Storage N series Introduction and Planning Guide. IBM System Storage N series storage systems are driven by NetApp Data ONTAP software. Some features described in the product software documentation are neither offered nor supported by IBM. Please contact your local IBM representative or reseller for further details. Information about supported features can also be found on the N series support website, which is accessed and navigated as described in Websites. IBM maintains pages on the World Wide Web where you can get the latest technical information and download device drivers and updates. The following web pages provide N series information: v v v v A listing of currently available N series products and features can be found at the following web page: The IBM System Storage N series support website requires users to register in order to obtain access to N series support content on the web. To understand how the N series support web content is organized and navigated, and to access the N series support website, refer to the following publicly accessible web page: This web page also provides links to AutoSupport information as well as other important N series product resources. IBM System Storage N series products attach to a variety of servers and operating systems. To determine the latest supported attachments, go to the IBM N series interoperability matrix at the following web page: For the latest N series hardware product documentation, including planning, installation and setup, and hardware monitoring, service and diagnostics, see the IBM N series Information Center at the following web page: Getting information, help, and service If you need help, service, or technical assistance or just want more information about IBM products, you will find a wide variety of sources available from IBM to assist you. This section contains information about where to go for additional information about IBM and IBM products, what to do if you experience a problem with your IBM N series product, and whom to call for service, if it is necessary. The following applies in Taiwan: xx IBM System Storage: N6000 Series Hardware and Service Guide

23 IBM Taiwan Product Service Contact Info: IBM Taiwan Corporation 3F, No 7, Song Ren Rd., Taipei Taiwan Tel: Before you call Before you call, make sure you have taken these steps to try to solve the problem yourself: v v v v Check all cables to make sure they are connected. Check the power switches to make sure the system is turned on. Use the troubleshooting information in your system documentation and use the diagnostic tools that come with your system. Refer to the IBM support website for information on known problems and limitations. Using the documentation The latest versions of N series software documentation, including Data ONTAP and other software products, are available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Current N series hardware product documentation is shipped with your hardware product in printed documents or as PDF files on a documentation CD. For the latest N series hardware product documentation PDFs, go to the IBM N series support website. Hardware documentation, including planning, installation and setup, and hardware monitoring, service, and diagnostics, is also provided in an IBM N series Information Center at the following web page: Hardware service and support You can receive hardware service through IBM Integrated Technology Services. Visit the following web page for support telephone numbers: Firmware updates IBM N series product firmware is embedded in Data ONTAP. As with all devices, it is recommended that you run the latest level of firmware. Any firmware updates are posted to the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Note: If you do not see new firmware updates on the IBM N series support website, you are running the latest level of firmware. About this document xxi

24 Verify that the latest level of firmware is installed on your machine before contacting IBM for technical support. Terminology and conventions used in this document This guide uses the following terminology, command conventions, format conventions and keyboard conventions. In this document, the term gateway describes IBM N series storage systems that have been ordered with gateway functionality. Gateways support various types of storage, and they are used with third-party disk storage systems. In this case, disk storage for customer data and the RAID controller functionality is provided by the back-end disk storage system. A gateway might also be used with disk storage expansion units specifically designed for the IBM N series models. The term filer describes IBM N series storage systems that either contain internal disk storage or attach to disk storage expansion units specifically designed for the IBM N series storage systems. Filer storage systems do not support using third-party disk storage systems. The terms system or storage system refer to either a gateway by itself or a filer, either by itself or with additional disk drives. Active/active and high-availability terms active/active configuration In the Data ONTAP 7.2 and 7.3 release families, refers to a pair of storage systems (sometimes called nodes) configured to serve data for each other if one of the two systems stops functioning. Also sometimes referred to as active/active pairs. cluster In the Data ONTAP 7.1 release family, refers to a pair of storage systems (sometimes called nodes) configured to serve data for each other if one of the two systems stops functioning. In Data ONTAP 8.x 7-Mode, a cluster is a group of connected nodes (storage systems) that share a global namespace and that you can manage as a single virtual server or multiple virtual servers, providing performance, reliability, and scalability benefits. HA (high availability) In Data ONTAP 8.x 7-Mode, the recovery capability provided by a pair of nodes (storage systems), called an HA pair, that are configured to serve data for each other if one of the two nodes stops functioning. HA pair In Data ONTAP 8.x 7-Mode, a pair of nodes (storage systems) configured to serve data for each other if one of the two nodes stops functioning. Storage terms ACP Alternate Control Path is a protocol that enables Data ONTAP to manage and control the storage expansion unit storage subsystem using a separate network from the data path, so management communication is not dependent on the data path being intact and available. AT-FCX The controller module of the EXN1000 serial advanced technology attachment (SATA) storage expansion unit. xxii IBM System Storage: N6000 Series Hardware and Service Guide

25 Disk Any hard disk drive Disk shelf or shelf Any storage expansion unit containing hard disk drives. ESH The controller module of Fibre Channel disk storage expansion units. IOM The Input/Output module of SAS disk storage expansion units. Loop or Fibre-Channel Arbitrated Loop One or more daisy-chained Fibre Channel storage expansion units connected to an N series storage system. Node The system controller module that executes the software on an N series system. There is one node in the single-controller models; there are two nodes in active/active or high availability) models. In a dual-controller N6000 series system, the nodes are located one over the other. The top one is controller module A and the bottom one is controller module B, as labeled on the system chassis. Stack or SAS stack A stack of one or more SAS storage expansion units using IOM modules connected to the storage system. The maximum number of storage expansion units in a stack and the number of storage expansion unit stacks in a configuration are dependent on the type of storage system. Command conventions You can enter commands on the system console or from any client that can obtain access to the storage system using a Telnet session. In examples that illustrate commands executed on a UNIX workstation, the command syntax and output might differ, depending on your version of UNIX. Formatting conventions The following table lists different character formats used in this guide to set off special information. Formatting convention Type of information Italic type v Words or characters that require special attention. v v Placeholders for information you must supply. For example, if the guide requires you to enter the fctest adaptername command, you enter the characters fctest followed by the actual name of the adapter. Book titles in cross-references. Monospaced font v Command and daemon names. Bold monospaced font v v Information displayed on the system console or other computer monitors. The contents of files. Words or characters you type. What you type is always shown in lowercase letters, unless your program is case-sensitive and uppercase letters are necessary for it to work properly. About this document xxiii

26 Keyboard conventions This guide uses capitalization and some abbreviations to refer to the keys on the keyboard. The keys on your keyboard might not be labeled exactly as they are in this guide. What is in this guide... hyphen (-) Enter type enter What it means... Used to separate individual keys. For example, Ctrl-D means holding down the Ctrl key while pressing the D key. Used to refer to the key that generates a carriage return, although the key is named Return on some keyboards. Used to mean pressing one or more keys on the keyboard. Used to mean pressing one or more keys and then pressing the Enter key. How to send your comments Your feedback is important in helping us provide the most accurate and high-quality information. If you have comments or suggestions for improving this document, send us your comments by to starpubs@us.ibm.com. Be sure to include the following: v Exact publication title v Publication form number (for example, GC ) v Page, table, or illustration numbers v A detailed description of any information that should be changed xxiv IBM System Storage: N6000 Series Hardware and Service Guide

27 Chapter 1. Preparing for the installation This chapter provides an overview of the entire IBM System Storage N6000 series system installation process, hardware specifications, and the appropriate documentation references for the procedures. Refer to the Installation and Setup Instructions that came with your system for further information about installing your equipment. This chapter discusses the following topics: v Required manuals, tools and equipment v Handling static-sensitive devices v Planning and organizing the installation on page 2 Required manuals, tools and equipment You need the following manuals in addition to this manual. Data ONTAP publications are available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. v Installation and Setup Instructions shipped with your system and storage expansion units v Data ONTAP Software Setup Guide for your version of Data ONTAP Attention: Before you begin your installation, print and complete a configuration worksheet for your storage system to gather the information that the software setup process requires. The configuration worksheet is provided in the Data ONTAP Software Setup Guide for your version of Data ONTAP. v Data ONTAP Active/Active Configuration Guide or Data ONTAP High Availability Configuration Guide for your version of Data ONTAP, if applicable You need to supply the following tools and equipment: v Ethernet LAN cables v Fibre Channel cables v Console (for example, a PC or laptop) with a serial port v #2 Phillips screwdriver and slotted screwdriver v Grounding leash and ESD strap v 7-mm nut driver (required if removing or installing storage expansion units) Handling static-sensitive devices CAUTION: This system uses electronic components that are sensitive to static electricity. Static discharge from your clothing or other fixtures around you can damage these components. Put on an antistatic ESD strap and grounding leash to free yourself of static electricity before touching any electronic components. Attention: Static electricity can damage electronic devices and your system. To avoid damage, keep static-sensitive devices in their static-protective packages until you are ready to install them. Copyright IBM Corp. 2008,

28 To reduce the possibility of electrostatic discharge (ESD), observe the following precautions: v v v v v v Limit your movement. Movement can cause static electricity to build up around you. Handle the device carefully, holding it by its edges or its frame. Do not touch solder joints, pins, or exposed printed circuitry. Do not leave the device where others can handle and possibly damage the device. While the device is still in its static-protective package, touch it to an unpainted metal part of the system unit for at least two seconds. This drains static electricity from the package and from your body. Remove the device from its package and install it directly into your system unit without setting it down. If it is necessary to set the device down, place it in its static-protective package. Do not place the device on your system unit cover or on a metal table. Take additional care when handling devices during cold weather because heating reduces indoor humidity and increases static electricity. Planning and organizing the installation This section identifies the shipment contents and the rules and regulations you need to observe for the proper installation of your system. It also provides an overview of the entire system installation process and the appropriate documentation references for the procedures. For detailed information, see the following topics: v Telecommunication regulatory statement v Hardware specifications v Checking shipment package contents on page 8 v Rules for installing the system in a rack on page 9 v Guide to the installation process on page 10 Telecommunication regulatory statement This product may not be certified in your country for connection by any means whatsoever to interfaces of public telecommunications networks. Further certification may be required by law prior to making any such connection. Contact an IBM representative or reseller for any questions. Hardware specifications The following table lists the characteristics and requirements for your hardware. Note: Operating at the extremes of the environmental requirements might increase the risk of device failure kg ( lbs) svc IBM System Storage: N6000 Series Hardware and Service Guide

29 DANGER The weight of this part or unit is between 44 and 54 kg (97 and 118 lb). It takes three persons to safely lift this part or unit. (C010) Table 1. N6000 series physical characteristics Weight v A10: 44.4 kg (97.8 lb) Rack units Height Width Depth v A20: 52.2 kg (115 lb) v A11: 44.9 kg (99 lb) v A21: 53.2 kg (117.3 lb) v A12: 44.8 kg (98.8 lb) v A22: 53.1 kg (117 lb) 6U 25.9 cm (10.2 in.) 44.7 cm (17.6 in.) 61.7 cm (24.3 in.) without cable tray 71.3 cm (28.1 in.) with cable tray Table 2. N6000 series clearance dimensions Front-cooling All versions 15.2 cm (6 in.) Rear-cooling All versions 30.5 cm (12 in.) Front-maintenance All versions 76.2 cm (30 in.) Rear-maintenance All versions 91.4 cm (36 in.) Table 3. N6000 series environmental requirements Operating temperature maximum range Operating temperature recommended range Nonoperating temperature range 50 F to 104 F (10 C to 40 C) 68 F to 77 F (20 C to 25 C) -40 F to 149 F (-40 C to 65 C) Relative humidity 20 to 80% noncondensing Recommended operating temperature relative humidity 40 to 55% range Maximum wet bulb temperature 28 C (82 F) Maximum altitude 3050 m (10,000 ft.) Acoustic level C C The following tables list the maximum electrical power for the N6000 series systems and the electrical requirements for different configurations of the N6000 series systems. Chapter 1. Preparing for the installation 3

30 Table 4. N6000 series systems maximum electrical power System Maximum electrical power N6000 series systems V ac, 12-8 A, Hz In the following tables, Worst-case indicates a system running with one PSU and high fan speed. Typical Per PSU/System, two PSUs indicates a system running one PSU on one circuit and a system running two PSUs on two circuits. Table 5. N6040 electrical requirements one controller module Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 6. N6040 electrical requirements two controller modules Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 7. N6040 electrical requirements with two PAM or Flash Cache module cards one controller module Input voltage Input current measured, A 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical System, two PSUs 4 IBM System Storage: N6000 Series Hardware and Service Guide

31 Table 7. N6040 electrical requirements with two PAM or Flash Cache module cards one controller module (continued) Input voltage Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 8. N6040 electrical requirements with four PAM or Flash Cache module cards two controller modules Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 9. N6060 electrical requirements one controller module Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Chapter 1. Preparing for the installation 5

32 Table 10. N6060 electrical requirements two controller modules Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 11. N6060 electrical requirements with two PAM or Flash Cache module cards one controller module Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 12. N6060 electrical requirements with four PAM or Flash Cache module cards two controller modules Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs 6 IBM System Storage: N6000 Series Hardware and Service Guide

33 Table 13. N6070 electrical requirements one controller module Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 14. N6070 electrical requirements two controller modules Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Table 15. N6070 electrical requirements with four PAM or Flash Cache module cards one controller module Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz 100 to 120V 200 to 240V Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Chapter 1. Preparing for the installation 7

34 Table 16. N6070 electrical requirements with eight PAM or Flash Cache module cards two controller modules 100 to 120V 200 to 240V Input voltage Input current measured, A Input power measured, W Thermal dissipation, BTU/hr Input power frequency, Hz Worstcase, single PSU Per PSU Typical System, two PSUs Worstcase, single PSU Per PSU Typical to 60 System, two PSUs Checking shipment package contents Make sure that your shipment package includes the following items: v Single-controller system 1 single-controller system containing the power supplies and any options you ordered 1 console adapter cable, RJ-45 to DB-9 1 cable management tray 1 ESD wrist strap 1 serial null modem cable 1 set of IBM publications 2 power cords 4 SFPs There will also be envelopes with the software EULA and license keys. A rail kit for mounting the single-controller system in a standard IBM 19 inch rack may also be present. v Dual-controller (active/active or high availability) system 1 dual-controller system containing the power supplies and any options you ordered 2 console adapter cables, RJ-45 to DB-9 2 cable management trays 1 wrist ESD straps 2 serial null modem cables 1 set of IBM publications 2 power cords 8 SFPs There will also be envelopes with the software EULA and license keys. A rail kit for mounting the dual-controller (active/active or high availability) system in a standard IBM 19 inch rack may also be present. 8 IBM System Storage: N6000 Series Hardware and Service Guide

35 Rules for installing the system in a rack Attention: The rack installation instructions provided in this document and in the Installation and Setup Instructions for your N series product apply specifically to the installation of the N series product in an IBM 19-inch rack. IBM service personnel cannot install the N series product in a non-ibm rack. If the N series product is being installed in a non-ibm rack, the rails shipped with the N series product may or may not work with the non-ibm rack. Physical installation of the N series product in a non-ibm rack is the customer's responsibility. You need to observe the following rules and restrictions when installing an N6000 series system in a standard IBM 19-inch (48.26 cm) equipment rack with mounting rails: kg ( lbs) svc00168 DANGER The weight of this part or unit is between 44 and 54 kg (97 and 118 lb). It takes three persons to safely lift this part or unit. (C010) v v v v v You must work with two other people. CAUTION: The system is extremely heavy. To avoid injuring yourself or damaging the system, you must work with at least two other people when you install the system in the rack. Install the system at the bottom of your configuration, so that loops extend above your system. DANGER To avoid hazardous conditions due to uneven mechanical loading, always install the heaviest devices in the bottom of the rack cabinet. Always install servers and optional devices starting from the bottom of the rack cabinet. (R001 part 1 of 2) When installing storage expansion units in a rack, do not exceed the maximum storage limit for your system. Make sure that the ID on the back panel of each storage expansion unit matches the ID specified on its label. Always install the storage expansion units fully loaded. Do not remove disk drives to reduce the weight. Chapter 1. Preparing for the installation 9

36 Guide to the installation process The following table provides a guide to the filer installation process. Table 17. Filer installation process procedures Stage Procedure 1 Print and complete a configuration worksheet for each system node of your storage system to gather the information that the software setup process requires. 2 Install the system in a standard IBM 19-inch rack. 3 Connect the system to the IP (Internet Protocol) network. 4 Filer: Connect the storage system to storage expansion units. Gateway: Connect the system to the back-end storage. Attention: Before you begin your installation, print and complete a configuration worksheet for your storage system to gather the information that the software setup process requires. The configuration worksheet is provided in the Data ONTAP Software Setup Guide for your version of Data ONTAP. This guide, as well as other Data ONTAP documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. If you are configuring a storage system as part of a high-availability (or active/active) configuration, some information types must be unique for each storage system mode in the configuration, and some information types must be identical on both storage system nodes. If you have a high-availability (or active/active) configuration, IBM recommends that you print and complete two copies of the configuration worksheet, one for each system node. Refer to the Installation and Setup Instructions that came with your system or storage expansion unit for complete installation details. Note: The initial hardware installation of an N6000 series gateway is performed by IBM. Additional installation services can be performed by IBM through an IBM services offering. Contact your IBM representative for more information. Is the procedure required? Procedure is performed by... Filer Gateway For instructions, see... Yes Customer The configuration worksheet is provided in the Data ONTAP Software Setup Guide for your version of Data ONTAP, Yes Customer IBM The Installation and Setup Instructions for your system Yes Customer Customer Connecting your system to an IP network on page 13, or the Installation and Setup Instructions that came with your system Yes Customer n/a Connecting a filer to storage expansion units on page 15, or the Installation and Setup Instructions that came with your system Yes n/a Customer Refer to the documentation for your external storage for additional information. 10 IBM System Storage: N6000 Series Hardware and Service Guide

37 Table 17. Filer installation process procedures (continued) Stage Procedure 5 Connect the system to a power source. 6 Configure the system. 7 Connect the system to a third-party device. Is the procedure required? Procedure is performed by... Filer Gateway For instructions, see... Yes Customer Customer Connecting your system to a power source on page 15, or the Installation and Setup Instructions that came with your system Yes Customer Customer The Data ONTAP Software Setup Guide for your version of Data ONTAP, or the Installation and Setup Instructions that came with your system No Customer Customer Connecting your system to storage on page 15 Chapter 1. Preparing for the installation 11

38 12 IBM System Storage: N6000 Series Hardware and Service Guide

39 Chapter 2. Connecting an N6000 series system Handling fiber-optic cables This chapter describes how to connect an N6000 series system in the following topics: v Handling fiber-optic cables v Connecting your system to an IP network v Connecting your system to a power source on page 15 v Connecting your system to storage on page 15 v Connecting your system to an ASCII terminal console on page 20 Before you use fiber-optic cables, read the following precautions. Attention: To avoid damage to the fiber-optic cables, follow these guidelines: v Do not route the cable along a folding cable-management arm. v When attaching to a device on slide rails, leave enough slack in the cable so that it does not bend to a radius of less than 38 mm (1.5 in.) when extended or become pinched when retracted. v Route the cable away from places where it can be snagged by other devices in the rack cabinet. v Do not overtighten the cable straps or bend the cables to a radius of less than 38 mm (1.5 in.). v Do not put excess weight on the cable at the connection point. Be sure that the cable is well supported. CAUTION: Data processing environments can contain equipment transmitting on system links with laser modules that operate at greater than Class 1 power levels. For this reason, never look into the end of an optical fiber cable or open receptacle. Connecting your system to an IP network Each node of your system connects to an IP network. If you have an active/active or high availability system, both nodes need to connect to the network. For information that describes how to connect your system, refer to the Installation and Setup Instructions that came with your system. The N6000 series system has two onboard Ethernet ports per node, labeled e0a and e0b, as shown in Figure 1 on page 14. Up to four Network Interface Cards (NICs) per node can be plugged into the PCI slots to provide additional Ethernet ports. The integrated Ethernet RJ-45 twisted-pair connectors are compatible with the IEEE Ethernet network 10/100/1000 BASE-TX link. When connecting to the Ethernet port, connect a twisted-pair (CAT-5 or better) cable to the RJ-45 Ethernet port located on the back of the system drawer. If you are connecting to a copper NIC, use RJ-45 CAT-5 or better copper cables. Copyright IBM Corp. 2008,

40 If you are connecting to a fiber NIC, use (50- or 62.5-micron) fiber-optic cables with LC connectors. e0a e0b! 0a 0b 0c 0d n6hw014 Figure 1. Onboard port locations (Ethernet) For information about monitoring the Ethernet port LEDs, see the IBM System Storage N Series Platform Monitoring Guide. About the e0m interface The e0m is the preferred Data ONTAP interface for the management LAN for environments that use dedicated LANs to isolate management traffic from data traffic. You use the e0m interface for Data ONTAP administrative tasks. See Figure 2. The e0m interface is separate from the RLM interface, but both interfaces sit behind the RJ-45 port (labeled with a wrench icon). This port is called the Management port and connects to an internal Ethernet switch, which provides connectivity to both interfaces. It is recommended that you configure both interfaces. Management LAN Data LAN Ethernet switch e0a e0b RLM e0m Data ONTAP Storage controller n6hw015 Figure 2. e0m interface See the Data ONTAP System Administration Guide for more information. 14 IBM System Storage: N6000 Series Hardware and Service Guide

41 Connecting your system to a power source The N6000 series systems are shipped with redundant power supplies, referred to as PSU1 and PSU2. Each power supply has its own AC power cord. You should have separate circuit breakers for each power supply to ensure power redundancy. For information on connecting your N6000 series system to a power source, see the Installation and Setup Instructions that came with your system. Connecting your system to storage The N6000 series system has four onboard Fibre Channel ports per node, labeled 0a thru 0d, as shown in Figure 3. Up to four HBAs per node can be plugged into the PCI slots to provide additional Fibre Channel ports. Attach the (50- or 62.5-micron) fiber-optic cables with LC connectors to the Fibre Channel ports. Note: SFPs must be firmly seated in the Fibre Channel ports before making connections. e0a e0b! 0a 0b 0c 0d n6hw014 Figure 3. Onboard port locations (Fibre Channel) Connecting a filer to storage expansion units You must connect at least one storage expansion unit per node to your N6000 series filer. A single-controller filer (A10/A11/A12) must be connected to at least one storage expansion unit. A dual-controller (active/active or high availability) filer (A20/A21/A22) must be connected to at least two storage expansion units. Fiber-optic cables must be used for the connection from the filer to the first storage expansion unit. Attention: Make sure that all storage expansion unit (1Gb/2Gb/4Gb) switches are set to the 2Gb or 4Gb (if supported) position. If necessary, refer to the documents that came with the storage expansion unit for information about checking and changing the switch setting. For information that describes how to connect your N6000 series filer using the onboard Fibre Channel ports to storage expansion units, see the Installation and Setup Instructions that came with your system. Attention: If you are using optional adapter cards instead of the onboard Fibre Channel ports to connect your N6000 series filer to storage expansion units, see the cabling instructions described in Cabling your system using Fibre Channel expansion adapters on page 16. Dual-path Fibre Channel cabling is supported for N6000 series filers. Dual-path Fibre Channel cabling is designed to improve reliability, availability and serviceability of the storage expansion units attached to the storage controller by creating two redundant paths from each storage controller to each loop of the Chapter 2. Connecting an N6000 series system 15

42 storage expansion units. For more information about using dual-path Fibre Channel cabling, see the Installation and Setup Instructions that came with your system. Cabling your system using Fibre Channel expansion adapters The illustrations in this section provide an example of cabling for a dual-node active/active or high availability N6000 series filer SAN configuration with four FCP host-side SAN connections and two loops of disk storage expansion units, each with dual-path (also called multipath) Fibre Channel cabling to the disk storage expansion units in each loop. The cabling shown in this example uses both onboard Fibre Channel ports and Fibre Channel expansion adapters for connections to expansion unit storage. Attention: If you are connecting to stacks of SAS storage expansion units, refer to the cabling instructions provided in the IBM System Storage N series Universal SAS and ACP Cabling Guide. For additional information about optional adapter cards, see the appropriate "Optional adapter cards" appendix in the IBM System Storage N series Introduction and Planning Guide. This guide, as well as other N series documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Important: If you are not using the onboard ports for storage, then you must set your onboard ports to Target mode, as described in the IBM System Storage N series Data ONTAP Block Access Management Guide for your version of Data ONTAP. Note: 1. The illustrations in the following sections show N6000 series storage system connections to EXN4000 or EXN2000 storage expansion units. In EXN1000 storage expansion units, the positions of the In and Out ports are reversed from those of the EXN4000 or EXN The ESH4 and ESH2 are self-terminating and do not have a terminate switch. The AT-FCX is self-terminating as long as no cable is plugged into the Out port of the last storage expansion unit. 3. Terminate unused Fibre Channel ports with Fibre Channel loopback plugs. Step 1: Connecting the storage controller to the Fibre Channel SAN: Connect Fibre Channel ports 0b and 0d of each storage controller to separate (dual-fabric) SAN switches, as shown in Figure 4 on page IBM System Storage: N6000 Series Hardware and Service Guide

43 Loop 1 Disk EXN 1 PSU Module A OUT IN IN OUT Module B PSU Loop 2 Disk EXN 1 PSU IN OUT OUT IN PSU Controller A PSU 0a 0b 0c 0d Controller B PSU 0a 0b 0c 0d n6hw016 Figure 4. Cabling ports 0b and 0d to switches 2. Connecting the storage controllers to the disk storage expansion units: Using Fibre Channel ports 0a and 0c on each storage controller, connect the storage controllers to each of the two loops of disk storage expansion units as follows. 1. Connect upper storage controller A Fibre Channel port 0a to the FC IN port of the upper ESH4 (or AT-FCX) controller in the first disk storage expansion unit of loop Connect lower storage controller B Fibre Channel port 0a to the FC IN port of the lower ESH4 (or AT-FCX) controller in the first disk storage expansion unit of loop Connect lower storage controller B Fibre Channel port 0c to the FC IN port of the upper ESH4 (or AT-FCX) controller in the first disk storage expansion unit of loop Connect upper storage controller A Fibre Channel port 0c to the FC IN port of the lower ESH4 (or AT-FCX) controller in the first disk storage expansion unit of loop 2. Figure 5 on page 18 shows the Fibre Channel connections from the onboard Fibre Channel ports on the N6000 series active/active or high availability system to storage expansion units. Chapter 2. Connecting an N6000 series system 17

44 Loop 1 Disk EXN 1 PSU Module A OUT IN IN OUT Module B PSU Loop 2 Disk EXN 1 PSU IN OUT OUT IN PSU Controller A PSU 0a 0b 0c 0d Controller B PSU 0a 0b 0c 0d N6hw017 Figure 5. Cabling ports 0a and 0c to storage expansion units Step 3: Connecting disk storage expansion units to disk storage expansion units: If there are more than one disk storage expansion units in each loop, connect the disk storage expansion units together as follows. For each storage expansion unit in the loop: 1. Connect the FC OUT port of the upper ESH4 (or AT-FCX) controller to the FC IN port of the upper ESH4 (or AT-FCX) on the next disk storage expansion unit in the loop. 2. Connect the FC OUT port of the lower ESH4 (or AT-FCX) controller to the FC IN port of the lower ESH4 (or AT-FCX) on the next disk storage expansion unit in the loop. Step 4. Connecting the last disk storage expansion unit in each loop back to the storage controller (dual-path Fibre Channel cabling): If implementing dual-path Fibre Channel cabling, connect the last disk storage expansion unit in each loop back to the other storage controller, as shown in Figure 6 on page 19. In this example, the dual-path cabling is achieved using expansion adapter cards. 1. Connect the upper ESH4 (or AT-FCX) controller FC OUT port of the last disk storage expansion unit in loop 1 to a Fibre Channel port in the lower storage controller B (shown being connected to Fibre Channel port A in the optional expansion adapter in slot 4 of the lower storage controller B). 2. Connect the lower ESH4 (or AT-FCX) controller FC OUT port of the last disk storage expansion unit in loop 1 to a Fibre Channel port in the upper storage controller A (shown being connected to Fibre Channel port A in the optional expansion adapter in slot 4 of the upper storage controller A). 3. Connect the upper ESH4 (or AT-FCX) controller FC OUT port of the last disk storage expansion unit in loop 2 to a Fibre Channel port in the upper storage controller A (shown being connected to Fibre Channel port B in the optional expansion adapter in slot 4 of the lower storage controller A). 4. Connect the lower ESH4 (or AT-FCX) controller FC OUT port of the last disk storage expansion unit in loop 2 to a Fibre Channel port in the lower storage controller B (shown being connected to Fibre Channel port B in the optional expansion adapter in slot 4 of the upper storage controller B). 18 IBM System Storage: N6000 Series Hardware and Service Guide

45 Loop 1 Disk EXN 1 PSU Module A OUT IN IN OUT Module B PSU Loop 2 Disk EXN 1 PSU IN OUT OUT IN PSU Controller A PSU 0a 0b 0c 0d Controller B PSU 0a 0b 0c 0d n6hw018 Figure 6. Cabling expansion cards to storage expansion units for dual-path Fibre Channel cabling Connecting a gateway to external storage You must use fiber-optic cables to connect a gateway to external storage on a Fibre Channel SAN. See the Gateway Interoperability Matrix at the following web page for supported third-party storage for your N6000 series system: Refer to the documentation for your external storage for additional information. Connecting to a third-party device You can connect third-party devices to your system through an optical Fibre Channel interface using any Fibre Channel port on the back of the chassis. N6000 series systems only support third-party devices with an optical Fibre Channel interface. For supported devices for your N6000 series system, see the Interoperability Matrix on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Refer to the documentation that comes with the third-party device for connection information. Rules for connecting the third-party devices Observe the following rules for connecting the third-party devices: v Use a fiber-optic cable that is: Appropriate to the Fibre Channel connection on your N6000 series system Of an approved length for the third-party device Note: See the documentation for the third-party device. Chapter 2. Connecting an N6000 series system 19

46 v v An unsupported tape backup device might cause the system to halt. To verify support for your third-party device, check the Interoperability Matrix on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. For additional information about Fibre Channel cables, see the IBM N series support website. Connecting your system to an ASCII terminal console The ASCII terminal console enables you to monitor the boot process, helps you configure your system after it boots, and enables you to perform system administration. For information about ASCII terminal console wiring, DB-9 to RJ-45 console adapter pin connections, and connecting your system to an ASCII terminal console, see the IBM System Storage N series Introduction and Planning Guide. This guide, as well as other N series documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. 20 IBM System Storage: N6000 Series Hardware and Service Guide

47 Chapter 3. Configuring an N6000 series system This chapter describes how to configure an N6000 series system in the following topics: v Configuring the N6000 series system v Configuring the Fibre Channel port Configuring the N6000 series system Refer to the completed configuration worksheet for your storage system. The configuration worksheet is provided in the Data ONTAP Software Setup Guide for your version of Data ONTAP, available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. If you are configuring a storage system as part of a high-availability (or active/active) configuration, some information types must be unique for each storage system mode in the configuration, and some information types must be identical on both storage system nodes. If you have a high-availability (or active/active) configuration, IBM recommends that you print and complete two copies of the configuration worksheet, one for each system node. For information about how to boot your filer for the first time, refer to the Installation and Setup Instructions that shipped with your system. For gateway systems, initial startup and configuration tasks must be performed by IBM. For additional information about gateway systems, see the documentation listed in the "Bibliography" appendix of the IBM System Storage N series Introduction and Planning Guide. This guide, as well as other N series documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Configuring the Fibre Channel port Each system controller module provides four independent Fibre Channel ports, identified as 0a, 0b, 0c and 0d. SFPs must be firmly seated in all four ports before connecting cables. The Fibre Channel ports can operate in target or initiator mode. Fibre Channel ports do not support mixed initiator/target mode. The default mode for the ports is initiator mode. You do not need to configure the ports to use them in initiator mode. The Fibre Channel ports should be used in initiator mode to communicate with tape backup devices, such as in a TapeSAN backup configuration. Fibre Channel ports on HBAs cannot be configured in target mode. They can only be used in initiator mode. Copyright IBM Corp. 2008,

48 N6000 series system active/active or high availability configurations Active/active or high availability configurations must be cabled to switches that support public loop topology. To connect an N6000 series system to a fabric topology that includes switches that only support point-to-point topology, such as McDATA Director class switches, you must connect the active/active or high availability configuration to an edge switch and use this switch as a bridge to the fabric. For information about specific switch models supported and fabric configuration guidelines, see the IBM System Storage N series Fibre Channel and iscsi Configuration Guide for your version of Data ONTAP on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Configuring for initiator mode To configure the system back to initiator mode, complete the following steps. 1. Set the specified onboard ports to operate in initiator mode by entering the following command: fcadmin config -t initiator adapter where adapter is the port number. You can specify more than one port. Example: The following example sets onboard ports 0c and 0d to initiator mode. fcadmin config -t initiator 0c 0d 2. Reboot the system by entering the following command: reboot 3. Verify that the Fibre Channel ports are online and configured in the correct state for your configuration by entering the following command: fcadmin config Example: The following output example shows two ports configured as Fibre Channel targets and two ports configured as initiators. n6000a> fcadmin config Local Adapter Type State Status a target CONFIGURED online 0b target CONFIGURED online 0c initiator CONFIGURED online 0d initiator CONFIGURED online For information on converting the onboard ports to target mode and configuring your SAN, see the Data ONTAP Block Access Management Guide for your version of Data ONTAP. Configuring and using the RLM The following table provides a list of documents for configuring and using the Remote LAN Module (RLM). These documents are available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. 22 IBM System Storage: N6000 Series Hardware and Service Guide

49 Table 18. RLM configuration manuals Manual Title Installation and Setup Instructions that came with your system IBM System Storage N series Diagnostics Guide IBM System Storage N series Data ONTAP System Administration Guide for your version of Data ONTAP IBM System Storage N series Platform Monitoring Guide Information Provided Describes the system and RLM cabling. Lists and describes the diagnostic tests for a new or existing RLM. Describes RLM configuration and use. Lists RLM error messages and gives corrective action for the error. Chapter 3. Configuring an N6000 series system 23

50 24 IBM System Storage: N6000 Series Hardware and Service Guide

51 Chapter 4. Replacing N6000 series system devices This chapter describes how to replace parts in your N6000 series system. For information about monitoring the LEDs for your N6000 series system and optional adapter cards, as well as information about error messages and troubleshooting, refer to the IBM System Storage N series Platform Monitoring Guide. For diagnostic information about your system, see the IBM System Storage N series Diagnostics Guide. Note: With the exception of the CompactFlash card procedures, all the procedures in this chapter apply to systems running either Data ONTAP 7.x or Data ONTAP 8.x 7-Mode. If you are replacing a CompactFlash card, be sure to refer to the appropriate instructions for your version of Data ONTAP. This chapter discusses the following topics: v Replacing a CompactFlash card in N6000 series systems running Data ONTAP 7.x on page 27 v Replacing a CompactFlash card in a system running in Data ONTAP 8.0 (7-Mode) on page 38 v Replacing a CompactFlash card in a system running Data ONTAP 8.2 (Cluster Mode) on page 55 v Replacing a fan module in a system on page 71 v Installing or replacing the RLM in a system on page 73 v Replacing a power supply on a system on page 80 v Replacing the Real-Time Clock (RTC) battery in a system on page 82 v Replacing an NVRAM battery in a system on page 88 v Replacing a DIMM in a system on page 94 v Replacing PCI cards and risers in a system on page 100 Figure 7 shows the locations of components at the front of your N6000 series system. Fan fault LED Cam handle with tab release 6 Fan modules for Active/Active configurations Power Fault 3 Fan modules for single-controller configurations Controller A activity Controller B activity n6hw011 Figure 7. N6000 series system components - Front Copyright IBM Corp. 2008,

52 Figure 8 shows the locations of components at the rear of your N6000 series system. Thumbscrew and cam handle Cam handle with tab release 2 Power supplies with power and fault LEDs Controller module A 4 Expansion adapter slots 4 Onboard Fibre Channel ports Fault status LED Controller module B (for Active/Active configurations only) Console port Management port 2 Network ports n6hw010 Figure 8. N6000 series system components - Rear Figure 9 shows a view of the exterior components of your N6000 series system. Power supply Fan module Controller module n6hw013 Folding handles Cable management tray Figure 9. N6000 series exterior components Figure 10 on page 27 shows an exploded view of the components of your N6000 series system. 26 IBM System Storage: N6000 Series Hardware and Service Guide

53 DIMM cover Controller module release tab System memory DIMMs RLM card CompactFlash card RTC battery NVRAM memory DIMM NVRAM battery Side panel NVRAM LED NVRAM LED PCIe riser Console, network and management ports NVRAM memory DIMM Controller system board n6hw012 Figure 10. N6000 series system components - Exploded view Using the IBM N series support website The IBM System Storage N series support website requires users to register in order to obtain access to N series support content on the web. To understand how the N series support web content is organized and navigated, and to access the N series support website, refer to the following publicly accessible web page: This web page also provides links to AutoSupport information as well as other important N series product resources. Replacing a CompactFlash card in N6000 series systems running Data ONTAP 7.x Note: This section contains procedures for two different Data ONTAP 7.x configurations. You must use the correct procedure for your configuration: v You can use this procedure only with systems running Data ONTAP 7.x. v v If you have a single-controller configuration, see Replacing a CompactFlash card in a single-controller system running Data ONTAP 7.x on page 28. If you have an active/active configuration, see Replacing a CompactFlash card in an active/active configuration running Data ONTAP 7.x on page 33. Chapter 4. Replacing N6000 series system devices 27

54 You might also need the following documentation to perform the identified procedures. Data ONTAP documentation is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. v IBM System Storage N series Data ONTAP Upgrade Guide, as applicable v v IBM System Storage N series Data ONTAP Commands: Manual Page Reference, if applicable IBM System Storage N series Data ONTAP System Administration Guide, if applicable These procedures are written with the assumption that you have access to at least one of the following: v A system on your network that you can use to netboot your system. v A PC with a card reader/writer and with access to the IBM N series support website. If your system does not meet these criteria, contact IBM technical support. Verifying system requirements Software release information and downloads, including Data ONTAP software, firmware, and diagnostics upgrades, are available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Replacing a CompactFlash card in a single-controller system running Data ONTAP 7.x This procedure describes how to replace a CompactFlash card in a single-controller configuration running Data ONTAP 7.x. If your system is in an active/active configuration running Data ONTAP 7.x, use the procedure in Replacing a CompactFlash card in an active/active configuration running Data ONTAP 7.x on page 33. This section describes how to perform the following tasks: v Removing the CompactFlash card on page 29 v Installing the CompactFlash card on page 30 v Placing the system files on the CompactFlash card on page 30 v Updating the CompactFlash card on page 32 v Completing the replacement process on page IBM System Storage: N6000 Series Hardware and Service Guide

55 Removing the CompactFlash card Figure 11. CompactFlash card removal 1. Retrieve and save the system environment variables, if possible, by completing the following substeps: a. Bring the system down to the boot loader prompt. v If the system is running, reboot it, press Ctrl-C when prompted by the system, and then go to substep 1b. v If the system is at the boot loader prompt, go to substep 1b. b. Display the system environment variables by entering the following command: printenv c. Copy the environment variable values. You need these variable values for the replacement CompactFlash card. Note: You do not need to retrieve the Fibre Channel port settings for a stand-alone configuration. Fibre Channel port settings are stored in the boot environment and also on disk in the system root volume. When you reboot the system, Fibre Channel settings are restored from the saved settings. 2. Turn off the power supplies, unplug both power cords from the power source, and then remove the power cords. 3. While grounded, remove the controller module from the chassis, as described in Opening the system on page Check the NVRAM memory LEDs on the controller module board. If the LEDs are flashing red, there is content in the NVRAM that has not been saved to disk. Complete the following substeps: a. Reinstall the controller module and reconnect the power supplies to the power source. b. Reboot the controller module. Chapter 4. Replacing N6000 series system devices 29

56 c. Repeat Steps 1 and 2. v If the NVRAM memory LEDs are no longer flashing red, go to step 5. v If repeated attempts to cleanly shut down the controller module fail, remove the controller module and proceed to step 5.However, be aware that you might lose any data that was not saved to disk. 5. Turn the controller module so that the cables are facing away from you, and then locate the CompactFlash card inside and on the left of the controller module. 6. Grasp the CompactFlash card and lift it straight up and out of the socket. 7. Go to Installing the CompactFlash card. Installing the CompactFlash card Attention: If you are copying the system files to the CompactFlash card using a PC or laptop with a card writer, you must copy the system files to the CompactFlash card prior to installing it in the controller module. See Transferring the system files using a PC or laptop on page 36 for information. 1. While grounded, align the CompactFlash card with the edges of the CompactFlash card socket. Seat the CompactFlash card by firmly pushing it down into the CompactFlash socket. The CompactFlash card should be squarely seated and should not move. Reseat the CompactFlash card, if necessary. 2. Reinstall the controller module: a. Close the DIMM cover and side panel, if necessary. b. Align the end of the controller module with the opening in the chassis, if necessary, and then gently push the controller module halfway into the chassis. The cam handle begins to engage when the controller module is properly seated all the way inside the system. c. Reinstall the cable management tray and recable the system, as needed. Note: If you are recabling a gateway system, see the Gateway Installation Requirements and Reference Guide. This guide, as well as other Data ONTAP documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. d. Tighten the thumbscrew beneath the cam handle. 3. Go to Placing the system files on the CompactFlash card. Placing the system files on the CompactFlash card You can transfer system files to the new CompactFlash card using netboot or a PC or laptop. The method you select depends on the following situations: v v Use netboot when you cannot copy the system files to the CompactFlash card because you do not have access to a card reader/writer but can access a network server. Use a PC or laptop when you can access a card reader/writer but you can only take the CompactFlash card to the system; you cannot leave with anything you take to the system. For example, you are entering a secured location and must leave behind whatever you take into the location. This section describes how to perform the following tasks: v Transferring the system files using netboot on page 31 v Transferring the system files using a PC or laptop on page IBM System Storage: N6000 Series Hardware and Service Guide

57 Note: This procedure is written with the assumption that you have access to a PC running Windows XP or later and have a zip program. Transferring the system files using netboot 1. Per the recommendations in the IBM System Storage N series Data ONTAP Upgrade Guide, place the system files on the server you use for netbooting. You can copy the system files from the system boot directory, at /etc/boot/netapp-x86-64, or download the file <rlse>_netboot.q (where <rlse> is the Data ONTAP release) from the IBM N series support website, which is accessed and navigated as described in Websites on page xx. 2. Turn on your system and press Ctrl-C to stop the boot process at the boot loader prompt. 3. Configure your network connection, if needed. v If you have DHCP running on your network, enter the following command at the prompt: ifconfig e0a -auto v If you do not have DHCP running, configure the connection by entering the following command at the prompt: ifconfig e0a -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain filer_addr is the IP address of the system. netmask is the network mask of the system. gateway is the gateway for the system. dns_addr is the IP address of a name server on your network. dns_domain is the DNS domain name. 4. Enter the following command at the boot loader prompt: netboot URL URL is the location of the remote system files. It can be either an HTTP or a TFTP network path. 5. Go to Updating the CompactFlash card on page 32. Transferring the system files using a PC or laptop 1. Download <rlse>_setup_q.exe to your PC. rlse is the Data ONTAP release you are using. You can download the system files from the IBM N series support website, which is accessed and navigated as described in Websites on page xx. On the IBM N series support website, the Data ONTAP Matrix provides a current list and history of Data ONTAP releases available by release family and supported storage systems. You can also copy the system files from the system boot directory of another system at /etc/boot/netapp-x Contact IBM technical support if you cannot get the system files from these sources. Attention: Make sure that you download the correct file, designated for PC administration, and that it is the same version of Data ONTAP as what is on your system. 2. Extract the contents of <rlse>_setup_q.exe to a temporary folder on your PC. 3. Insert the CompactFlash card into the CompactFlash card reader. Chapter 4. Replacing N6000 series system devices 31

58 Attention: The CompactFlash card is pre-formatted. Do not format the CompactFlash card. 4. Create a folder called X86_64 in the root partition of the CompactFlash card, and then create a subfolder called KERNEL in the X86_64 folder. 5. Copy netapp-x86-64 into the KERNEL folder in Windows Explorer. 6. Rename the netapp-x86-64 image to Primary.KRN. 7. Install the CompactFlash card with the kernel image into the system. 8. Reinstall the controller module, recable it, and turn on the power. 9. The controller will boot when power is turned on. If necessary, boot Data ONTAP by entering the following command at the LOADER prompt: boot_ontap 10. Go to Updating the CompactFlash card. Updating the CompactFlash card 1. Download the system files to the CompactFlash card by entering the following command: download Attention: If the download fails with the following error message, you must run setup to reinstall the proper files on the system that enable you to execute the download command on the CompactFlash card: Failed to open download script file /etc/boot/x86-64/kernel_1024.cmds: No such file download.requestdoneerror:error]: Operator requested download failed. 2. Check the version of the image by entering the following command: version -b Make sure that the kernel version on the CompactFlash card is the same version was originally on the system. 3. Reboot the system by entering the following command: reboot 4. Go to Restoring environment variables. Restoring environment variables 1. Check the version of the image by entering the following command: version -b Make sure that the BIOS and Diagnostic images are the correct version. If they are not, go to the IBM N series support website, which is accessed and navigated as described in Websites on page xx, and follow the installation instructions to install the version you need. 2. Halt the controller by entering the following command: halt 3. Display the default environment variables by entering the following command: printenv 4. Compare the default environment variables with the list you made in Removing the CompactFlash card on page 29. If the variable values are the same, go to step 5. If the variable values are different or need modification, complete the following steps: 32 IBM System Storage: N6000 Series Hardware and Service Guide

59 a. Set the values of the individual variables by entering the following command for each changed variable: setenv variable_name variable_value b. Verify the values of the variables by entering the following command from the boot loader prompt: printenv 5. Boot Data ONTAP after you modify all variables by entering the following command: boot_ontap 6. Go to Completing the replacement process. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Replacing a CompactFlash card in an active/active configuration running Data ONTAP 7.x This procedure describes how to nondisruptively replace the CompactFlash card in an active/active configuration running Data ONTAP 7.x. If your system is a single-controller system, use the procedure in Replacing a CompactFlash card in a single-controller system running Data ONTAP 7.x on page 28. Note: The following steps may be disruptive and are recommended to be preformed only during a maintenance window. This section describes how to perform the following tasks: v Preparing for the CompactFlash card replacement v Removing the CompactFlash card on page 34 v Installing the replacement CompactFlash card on page 34 v Updating the CompactFlash card on page 36 v Restoring environment variables and onboard Fibre Channel port configurations on page 37 v Completing the replacement process Preparing for the CompactFlash card replacement 1. Check the status of the active/active configuration and take over the target node. v If you can connect to the target node through normal means, complete the following substeps: a. Check that the active/active configuration is enabled by entering the following command: cf status If it is enabled, go to substep b. If it is not enabled, enable it by entering the following command: cf enable b. Take over the target node by entering the following command from the partner console: cf takeover Chapter 4. Replacing N6000 series system devices 33

60 c. Stop the target node from rebooting by pressing Ctrl-C when you see the following message: Starting AUTOBOOT press Ctrl-C to abort d. Go to step 2. v If you cannot connect to the target node through normal means and the node is receiving power, complete the following substeps: a. Connect a console directly to the target node through the marked console port. Follow the manufacturer's installation instructions for configuration. The boot loader prompt should be accessible after the console connection is established. If it is not, contact technical support. b. Boot the target node from the backup system files on the CompactFlash card. You might need to rename the backup system file so that it boots as the primary file. c. Return to the partner node and enable clustering by entering the following command: cf enable d. Take over the target node by entering the following command from the partner console: cf takeover e. Stop the target node from rebooting by pressing Ctrl-C when you see the following message: Starting AUTOBOOT press Ctrl-C to abort f. Go to step At the boot loader prompt, display the environment variables for the target node by entering the following command: printenv 3. Copy the values for the variables and save them for later use when you restore the environment variables on the replacement CompactFlash card. 4. Go to the partner console and enter the following command: partner fcadmin config 5. Copy the values for the onboard Fibre Channel settings and save them for later use when you restore the settings on the replacement CompactFlash card. 6. Go to Removing the CompactFlash card. Removing the CompactFlash card 1. While grounded, shut down the controller module and remove it from the chassis, as described in Shutting down a node in an HA pair on page 73 and Opening the system on page Turn the controller module so that the cables are facing away from you and then locate the CompactFlash card inside and on the left of the controller module. 3. Grasp the CompactFlash card and lift it up and out of the socket. 4. Go to Installing the replacement CompactFlash card. Installing the replacement CompactFlash card You can transfer system files to the new CompactFlash card using netboot or a PC or laptop. The method you select depends on the following situations: v Use netboot when you cannot copy the system files to the CompactFlash card because you do not have access to a card reader/writer but can access a network server. 34 IBM System Storage: N6000 Series Hardware and Service Guide

61 v Use a PC or laptop when you can access a card reader/writer but you can only take the CompactFlash card to the system; you cannot leave with anything you take to the system. For example, you are entering a secured location and must leave behind whatever you take into the location. This section describes how to perform the following tasks: v Transferring the system files using netboot v Transferring the system files using a PC or laptop on page 36 Transferring the system files using netboot 1. Make sure that you are properly grounded. 2. Per the recommendations in the Data ONTAP Upgrade Guide, place the system files on the server you use for netbooting. You can copy the system files from the system boot directory, at /etc/boot/netapp-x86-64, or download the file <rlse>_netboot.q (where <rlse> is the Data ONTAP release) from the IBM N series support website, which is accessed and navigated as described in Websites on page xx. On the IBM N series support website, the Data ONTAP Matrix provides a current list and history of Data ONTAP releases available by release family and supported storage systems. 3. Align the CompactFlash card with the edges of the CompactFlash card slot. Seat the CompactFlash card by sliding it into the CompactFlash reader. The CompactFlash card should be squarely seated and should not move. Reseat the CompactFlash card, if necessary. 4. Reinstall the controller module into the chassis and recable it. Note: The node starts to reboot as soon as you insert it fully back into the chassis. 5. Press Ctrl-C to stop the boot process and go to the boot loader prompt. 6. Configure your network connection, if needed. If you have DHCP running on your network, enter the following command at the prompt: ifconfig e0a -auto If you do not have DHCP running, configure the connection by entering the following command at the prompt: ifconfig e0a -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain filer_addr is the IP address of the system. netmask is the network mask of the system. gateway is the gateway for the system. dns_addr is the IP address of a name server on your network. dns_domain is the DNS domain name. 7. Enter the following command at the boot loader prompt: netboot URL URL is the location of the remote system files from step 2. It can be either an HTTP or a TFTP network path. 8. On the failed node, at the Special Boot Menu, select Option 1 for normal boot. Wait until you receive the prompt: Waiting for Giveback (press Ctrl-C to abort wait). 9. On the partner node, enter the following command: cf giveback Chapter 4. Replacing N6000 series system devices 35

62 10. After the target node completes booting, go to Updating the CompactFlash card. Transferring the system files using a PC or laptop 1. Download <rlse>_setup_q.exe to your PC. rlse is the Data ONTAP release you are using. You can download the system files from the IBM N series support website, which is accessed and navigated as described in Websites on page xx. On the IBM N series support website, the Data ONTAP Matrix provides a current list and history of Data ONTAP releases available by release family and supported storage systems. You can also copy the system files from the system boot directory of another storage system at /etc/boot/netapp-x Contact IBM technical support if you cannot get the system files from these sources. Attention: Make sure that you download the correct file, designated for PC administration, and that it is the same version of Data ONTAP as what is on your system. 2. Extract the contents of <rlse>_setup_q.exe to a temporary folder on your PC. 3. Insert the CompactFlash card into the CompactFlash card reader. Attention: The CompactFlash card is pre-formatted. Do not format the CompactFlash card. 4. Create a folder called X86_64 in the root partition of the CompactFlash card, and then create a subfolder called KERNEL in the X86_64 folder. 5. Copy netapp-x86-64 into the KERNEL folder in Windows Explorer. 6. Rename the netapp-x86-64 image to Primary.KRN. 7. While grounded, install the CompactFlash card into the controller module. 8. Reinstall the controller module into the chassis and recable it. Note: The node starts to reboot as soon as you insert it back into the chassis. Make sure that you give back the node when prompted to do so. 9. The controller will boot when power is turned on. If necessary, boot Data ONTAP by entering the following command at the LOADER prompt: boot_ontap 10. Go to Updating the CompactFlash card. Updating the CompactFlash card 1. Download the system files to the CompactFlash card by entering the following commands: download Attention: If the download fails with the following error message, you must run setup to reinstall the proper files on the system that enable you to execute the download command on the CompactFlash card: Failed to open download script file /etc/boot/x86-64/kernel_1024.cmds: No such file [download.requestdoneerror:error]: Operator requested download failed. 2. Check the version of the image by entering the following command: version -b 36 IBM System Storage: N6000 Series Hardware and Service Guide

63 Make sure that the kernel version on the CompactFlash card is the same version was originally on the system. 3. Reboot the system by entering the following command: reboot 4. Press Ctrl-C to stop the system at the boot loader prompt, then go to Restoring environment variables and onboard Fibre Channel port configurations. Restoring environment variables and onboard Fibre Channel port configurations 1. Retrieve the environment variables you saved in Preparing for the CompactFlash card replacement on page Enter the following command for each variable you must reset: setenv variable value variable is the variable name. value is the setting you are assigning to the variable. 3. Boot Data ONTAP by entering the following command from the target node's console: boot_ontap 4. When the following message is displayed on the target node's console: Waiting for giveback...press Ctrl-C to abort wait enter the following command from the partner node's console: cf giveback 5. After giveback completes, from the target node's console, check the configuration of the onboard Fibre Channel ports by entering the following command: fcadmin config v v If the displayed information is the same as what you captured for the onboard Fibre Channel ports in step 5. of Preparing for the CompactFlash card replacement on page 33, then go to Completing the replacement process on page 33. If the displayed information is different from what you captured for the onboard Fibre Channel ports, complete the following substeps: a. Reboot the node and press Ctrl-C when Press Ctrl-C for special boot menu appears. b. Press Ctrl-C again when Press CTRL-C for Maintenance menu to release disks appears. c. Confirm disk release when prompted. d. Reset the Fibre Channel ports in one of three ways: To reset the target ports, enter the following command for each port: fcadmin config -t target adapter_name To reset the initiator ports, enter the following command for each port: fcadmin config -t initiator adapter_name To reset the ports to unconfigured, enter the following command for each port: fcadmin config -t unconfig adapter_name e. On the failed node, verify the values of the variables by entering the following command from the boot loader prompt: printenv Chapter 4. Replacing N6000 series system devices 37

64 6. On the failed node, boot Data ONTAP by entering the following command: boot_ontap 7. Go to Completing the replacement process on page 33. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Replacing a CompactFlash card in a system running in Data ONTAP 8.0 (7-Mode) Replacing a CompactFlash card running in 7-Mode involves shutting down the system, removing the old CompactFlash card, and transferring the system files to the new CompactFlash card. About this task This procedure is for systems running in 7-Mode. If your system is running clustered Data ONTAP, you cannot use this procedure and must use the procedure for clustered Data ONTAP instead. Shutting down a node operating in 7-Mode When performing maintenance on a system operating in 7-Mode, you must shut down the node. Depending on your system's configuration, you might also need to turn off the power supplies. About this task Your system's configuration determines whether you turn off your power supplies after shutting down the node: v v If you have two controller modules in the same chassis, you must leave the power supplies turned on to provide power to the healthy node. If you have one controller module in the chassis that is either part of an HA pair or in a stand-alone configuration, you must turn off the power supplies in the impaired node chassis. Procedure 1. Either shut down or perform a takeover of the impaired node, depending on your configuration: If the system is in... Then... A stand-alone configuration and is running A stand-alone configuration and is not running An HA pair 1. Enter the following command at the console: halt -t 0 2. Go to step 6. Go to step 6. Go to the next step. 2. Check the status of the impaired node by entering the following command from the healthy node's console: cf status 38 IBM System Storage: N6000 Series Hardware and Service Guide

65 3. Take one of the following actions, depending on the result of the cf status command: Status of the impaired node It has been taken over by the healthy node and is halted It has been taken over by the healthy node and is showing the Waiting for giveback... message It has taken over the healthy node The impaired node has not been taken over by the healthy node and is running Then Go to step 5. Go to the next step. 1. Resolve any issues on the healthy node that lead to the takeover. 2. Give back the healthy node using the cf giveback command from the impaired node. Enter the following command from the healthy node's prompt: cf takeover 4. Wait at least two minutes after takeover of the impaired node to ensure that the takeover completed successfully. 5. With the impaired node showing the Waiting for giveback message or halted, shut it down, depending on your configuration: Is the Remote LAN Module (RLM) configured? Yes No Then... Log in to the RLM and issue the following command: system power off At the impaired node's prompt, press Ctrl-C and respond Y to halt the node. 6. If in a stand-alone configuration, shut down the power supplies and unplug both power cords from the source. Opening the system To access components inside the controller module, you must open the system. Procedure 1. If you are not already grounded, properly ground yourself. 2. Unplug the system cables and SFPs from the controller module, as needed, and keep track of where the cables were connected. Leave the cables in the cable clips of the cable management tray so that when you reinstall the cable management tray, the cables are already aligned. 3. Grasp the cable management tray by the side, gently push one side of the tray so that the arm slides across and off the retaining pin, and then lift it off the back of the controller module and set it aside. 4. Loosen the thumbscrew on the cam handle. 5. Pull the cam handle downward and slide the controller module out of the system until it catches. Chapter 4. Replacing N6000 series system devices 39

66 6. Push in the release latch on the left side of the controller module and slide the controller module out of the system and set it on a stable, antistatic surface. Make sure that you support the bottom of the controller module with your free hand. Removing the CompactFlash card from the controller module You must remove the old or failed CompactFlash card from the target controller module in your system prior to installing the new or replacement CompactFlash card. Procedure 1. If you are not already grounded, properly ground yourself. 2. Remove any cabling from the controller module, if necessary. Make sure that you keep track of where the cables were connected into the controller module. 3. Remove the controller module and turn it so that you can see where the CompactFlash card is located Rear of the N6040, N6060, or N6070 controller module CompactFlash card 4. Grasp the edges of the CompactFlash card and gently lift it out of the socket. 5. Set the CompactFlash card aside. 6. Determine which method you will use to transfer the system files to the replacement boot device; using a PC or laptop with a card reader/writer, or using netboot. Installing the CompactFlash card After you remove the old CompactFlash card, you must transfer system files and restore configuration information to your replacement CompactFlash card using one of several methods. 40 IBM System Storage: N6000 Series Hardware and Service Guide

67 Before you begin Note: For systems running 7-Mode, IBM recommends using the nondisruptive procedure for HA pair running Data ONTAP and later. For systems running clustered Data ONTAP, IBM recommends using the nondisruptive procedure for HA pair running Data ONTAP 8.2 and later. Installing the CompactFlash card using a PC or laptop with a card reader/writer You can transfer the system files to your replacement CompactFlash card using a PC or laptop with a CompactFlash card reader/writer prior to installing the replacement boot device into your controller. Before you begin For copying system files to the CompactFlash card, you must have the following: v The blank replacement CompactFlash card you received from your provider v A PC or laptop with a CompactFlash reader/writer that is running Windows XP or later and that has a.zip program, such as WinZip, installed v Access to the IBM N series support website. This enables you to download the necessary system files for your platform and version of Data ONTAP running on it. Transferring the system files to a CompactFlash card operating in a 7-Mode configuration using a PC or laptop: You can transfer the system files to the replacement CompactFlash card operating in a 7-Mode configuration using a PC or laptop that has a CompactFlash card reader/writer. Procedure 1. Download the *.tgz system image file from the IBM N series support website to a temporary folder on your PC. The file you download is named <release_name>_q_image.tgz. 2. Insert the blank CompactFlash card into the CompactFlash card reader/writer and then access the boot device using Windows Explorer. Note: The CompactFlash card is preformatted. If you reformat the card, it should be in FAT-16 format only. 3. Create a folder in the root partition of the boot device, naming it x86_ Create a freebsd/ subfolder, and then create image1/ and image2/ subdirectories inside freebsd/. The directory structure should resemble the following: x86_64/ freebsd/ image1/ image2/ 5. Navigate to the folder containing the *.tgz file on the PC and click the *.tgz file. 6. Extract the files to the following locations on the CompactFlash card: v freebsd/image1 v freebsd/image2 Chapter 4. Replacing N6000 series system devices 41

68 7. Return to the open zip-file window that displays the contents of the *.tgz file that you want to install. 8. Uncompress and untar, if necessary, the contents of the file named diags.tgz to the root directory on the CompactFlash card. This creates the directory structure for the diagnostics utilities and firmware. 9. Verify the directory structure on the CompactFlash card. The directory structure should resemble the following: x86_64/ freebsd/ image1/ image2/ diag/ firmware/ common/ firmware/ 10. If the node or controller is in an HA pair and the partner took over the node, copy the backup system configuration file, varfs.tgz, from the target node's root volume. Note: If the node or controller is a stand-alone system or in an HA pair where the node was not taken over by the partner, skip this step and go to the next step. 42 IBM System Storage: N6000 Series Hardware and Service Guide

69 To copy the backup system configuration file using... Complete the following substeps... An NFS client 1. Determine the taken-over node's name by entering the following command from the partner's console: partner hostnamenode_name 2. Export the node's root volume from the partner by entering the following command: partner exportfs -p sec=sys,rw,anon=0,nosuid /vol/vol0 3. Go to the NFS client and create a directory for the taken-over node on the NFS client by entering the following command: mkdir /mnt/node_name 4. Mount the exported volume by entering the following command on the NFS client: mount node_name:/vol/vol0 /mnt/node_name 5. Verify the contents of the newly mounted volume by entering the following command on the NFS client: ls -al /mnt/node_name/etc/varfs.tgz -r-xr-xr-x 1 root root :34 /mnt/node_name/etc/ varfs.tgz 6. Copy the varfs.tgz file to a directory on the NFS client that you can access with the PC by entering the following command on the NFS client console: cp /mnt/node_name/etc/varfs.tgz /u/target_directory/varfs.tgz 7. Go to the PC and copy the varfs.tgz file to the freebsd/ directory. The directory structure should appear as follows: x86_64/ freebsd/ image1/ image2/ varfs.tgz diag/ firmware/ common/ firmware/ 8. Go to the next step. A CIFS client 1. Open Windows Explorer and enter the path to the partner controller by entering the following command in the navigation pane: \\partner_node_name\c$ 2. Open the /etc folder. 3. Copy the varfs.tgz file and paste it into target directory. 4. Go to the PC and copy the varfs.tgz file to the freebsd/ directory. The directory structure should appear as follows: x86_64/ freebsd/ image1/ image2/ varfs.tgz diag/ firmware/ common/ firmware/ 5. Go to the next step. 11. Close the.zip program. Chapter 4. Replacing N6000 series system devices 43

70 12. Right-click the drive letter that is mapped to the CompactFlash card, select Eject to remove the CompactFlash card from the PC or laptop, and then remove the CompactFlash card from the CompactFlash card reader/writer. Installing the CompactFlash card into the 7-Mode controller: After the system files are transferred to the replacement boot device, you must install it into the controller. Procedure 1. If you are not already grounded, properly ground yourself. 2. Turn the controller module so that you can locate the CompactFlash card socket. 3. Align the CompactFlash card with the CompactFlash card socket or connector, and then firmly push the CompactFlash card into the socket or connector. The CompactFlash card socket is keyed. If you feel resistance while pushing the card into the socket, check the orientation of the card. 4. Check the CompactFlash card to make sure that it is seated squarely and completely in the socket or connector. If necessary, remove the CompactFlash card and reseat it into the socket. 5. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system. 6. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 7. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, push the cam handle to the closed position, and then tighten the thumbscrew. 8. The next step depends on your system configuration: If your system has... One controller module in the chassis Two controller modules in the chassis Then... Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. The node already began rebooting; go to the next step. Rebooting the controller running 7-Mode: After copying the system files to the new CompactFlash card and installing it back into the controller, you must reboot the system. Procedure 1. Reboot the node, if necessary. If... AUTOBOOT is set on the node AUTOBOOT is not set or the node does not begin to reboot Then The next step depends on your system configuration: The node begins the reboot process. Enter the following command from the prompt: boot_ontap 44 IBM System Storage: N6000 Series Hardware and Service Guide

71 If your system is... In an HA pair and the partner has taken over the node In a stand-alone configuration or in an HA pair and you did not copy the varfs.tgz to the boot device Then... The node reboots normally and awaits giveback. Give back the node by entering the following command from the partner's console: cf giveback After mounting the root volume, the backup configuration data is restored to the CompactFlash boot device and the controller automatically reboots. In HA pairs, you need to perform a giveback when the node finishes rebooting. Complete the following substeps when the following menu is displayed: 1. Select the Update flash from backup config. from the displayed menu. 2. Enter Y when you see the following message: This will replace all flash-based configuration with the last backup to disks. Are you sure you want to continue?: y When you select this option, the system attempts to mount the root volume where the backup configuration data is located. Note: The login prompt is not displayed on the console until after the controller reboots. Installing the CompactFlash card and transferring system files nondisruptively using netboot in systems running Data ONTAP 8.0.x or 8.1 You can transfer the system files and restore configuration information to your HA pair nondisruptively by using netboot and the partner node. Before you begin v You must have the blank replacement CompactFlash card that you received from your provider. v You must have a network interface to the HA pair partner node. v You must have a separate IP address to use to netboot the target node. About this task The following procedure is written with the assumption that the target node has been taken over by the partner node and that the target node has booted to the boot environment prompt after the takeover has been completed. Attention: You can only use this nondisruptive procedure in an HA pair that is running Data ONTAP 8.0.x or 8.1.You must use the disruptive procedure if your system is running Data ONTAP 8.0. Procedure 1. Download and extract the file used for performing the netboot of your system: a. Download the appropriate netboot.tgz file for your platform from the IBM N series support website to a web-accessible directory. b. Change to the web-accessible directory. c. Extract the contents of the netboot.tgz file to the target directory by entering the following command: tar -zxvf netboot.tgz Chapter 4. Replacing N6000 series system devices 45

72 Your directory listing should contain the following directory: netboot/ 2. Download the image.tgz file from the IBM N series support website to the web-accessible directory. Your directory listing should contain the following file and directory: image.tgz netboot/ 3. If you are not already grounded, properly ground yourself. 4. Turn the controller module so that you can locate the CompactFlash card socket. 5. Align the CompactFlash card with the CompactFlash card socket or connector, and then firmly push the CompactFlash card into the socket or connector. The CompactFlash card socket is keyed. If you feel resistance while pushing the card into the socket, check the orientation of the card. 6. Check the CompactFlash card to make sure that it is seated squarely and completely in the socket or connector. If necessary, remove the CompactFlash card and reseat it into the socket. 7. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system. 8. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 9. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, push the cam handle to the closed position, and then tighten the thumbscrew. 10. The next step depends on your system configuration: If your system has... One controller module in the chassis Two controller modules in the chassis Then... Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. The node already began rebooting; go to the next step. 11. Set up the TFTP server on the partner node by entering the following commands on the partner node console: partner options tftpd.enable off partner options tftpd.rootdir /etc partner options tftpd.enable on 12. Depending on your network configuration, enter one of the following commands at the LOADER prompt: If you... Have DHCP enabled Then... Enter the following command: ifconfig e0a -auto 46 IBM System Storage: N6000 Series Hardware and Service Guide

73 If you... Do not have DHCP enabled Then... Enter the following command: ifconfig e0a -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain filer_addr is the IP address of the storage system. netmask is the network mask of the storage system. gateway is the gateway for the storage system. dns_addr is the IP address of a name server on your network. dns_domain is the Domain Name System (DNS) domain name. If you use this optional parameter, you do not need a fully qualified domain name in the netboot server URL; you need only the server s host name. Note: To netboot the node when your system is running in 7-Mode Data ONTAP, use an IP address that is not the management IP address for the target. If your system is running clustered Data ONTAP, you can use the management IP address. Note: Other parameters might be necessary for your interface. For details, use the help ifconfig command at the LOADER prompt. 13. At the boot environment prompt, enter the following command: netboot Select the Install new software first option from the displayed menu. This menu option downloads and installs the new Data ONTAP image to the CompactFlash card. If you are prompted to continue the procedure, enter y when prompted. 15. Enter y when prompted regarding non-disruptive upgrade or replacement of the software. 16. Enter the path to the image.tgz file when you see the following prompt: What is the URL for the package? Restore the backup configuration on the target node by entering y and then entering the IP address of the server containing the varfs.tgz file when you see the following screen: Chapter 4. Replacing N6000 series system devices 47

74 **************************************************************** * Restore Backup Configuration * * This procedure only applies to storage controllers that * * are configured as an HA pair. * * * * Choose Yes to restore the "varfs" backup configuration * * from the TFTP server. Refer to the Boot Device Replacement * * guide for more details. * * Choose No to skip the backup and recovery and return to the * * boot menu. * **************************************************************** Do you want to restore the backup configuration now? {yn} y Enter the IP address of the server: target_node.management_ip.address Checking network link... success. Checking route to host "target_node.management_ip.address"... success. Attempting to reach "target_node.management_ip.address"... success. Checking boot device file system... success. Mounting boot device to /tmp/mnt... success. Checking boot device mount point... success. Restoring backup configuration... Received bytes in 0.1 seconds Backup Configuration from $BACKUP_DATE successfully restored Installing the CompactFlash card and transferring system files nondisruptively using netboot in systems running Data ONTAP and later You can transfer the system files and restore configuration information to your HA pair nondisruptively by using netboot and the partner node. Before you begin v You must have the blank replacement CompactFlash card that you received from your provider. v You must have a network interface to the HA pair partner node. v You must have a separate IP address to use to netboot the target node. About this task The following procedure is written with the assumption that the target node has been taken over by the partner node and that the target node has booted to the boot environment prompt after the takeover has been completed. Attention: You can only use this nondisruptive procedure in an HA pair that is running Data ONTAP and later. Procedure 1. Download and extract the file used for performing the netboot of your system: a. Download the appropriate netboot.tgz file for your platform from the IBM N series support website to a web-accessible directory. b. Change to the web-accessible directory. c. Extract the contents of the netboot.tgz file to the target directory by entering the following command: tar -zxvf netboot.tgz Your directory listing should contain the following directory: netboot/ 48 IBM System Storage: N6000 Series Hardware and Service Guide

75 2. Download the image.tgz file from the IBM N series support website to the web-accessible directory. Your directory listing should contain the following file and directory: image.tgz netboot/ 3. If you are not already grounded, properly ground yourself. 4. Turn the controller module so that you can locate the CompactFlash card socket. 5. Align the CompactFlash card with the CompactFlash card socket or connector, and then firmly push the CompactFlash card into the socket or connector. The CompactFlash card socket is keyed. If you feel resistance while pushing the card into the socket, check the orientation of the card. 6. Check the CompactFlash card to make sure that it is seated squarely and completely in the socket or connector. If necessary, remove the CompactFlash card and reseat it into the socket. 7. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system. 8. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 9. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, push the cam handle to the closed position, and then tighten the thumbscrew. 10. The next step depends on your system configuration: If your system has... One controller module in the chassis Two controller modules in the chassis Then... Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. The node already began rebooting; go to the next step. 11. Depending on your network configuration, enter one of the following commands at the LOADER prompt: If you... Have DHCP enabled Then... Enter the following command: ifconfig e0a -auto Chapter 4. Replacing N6000 series system devices 49

76 If you... Do not have DHCP enabled Then... Enter the following command: ifconfig e0a -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain filer_addr is the IP address of the storage system. netmask is the network mask of the storage system. gateway is the gateway for the storage system. dns_addr is the IP address of a name server on your network. dns_domain is the Domain Name System (DNS) domain name. If you use this optional parameter, you do not need a fully qualified domain name in the netboot server URL; you need only the server s host name. Note: To netboot the node when your system is running in 7-Mode Data ONTAP, use an IP address that is not the management IP address for the target. If your system is running clustered Data ONTAP, you can use the management IP address. Note: Other parameters might be necessary for your interface. For details, use the help ifconfig command at the LOADER prompt. 12. At the boot environment prompt, enter the following command: netboot Select the Install new software first option from the displayed menu. This menu option downloads and installs the new Data ONTAP image to the CompactFlash card. If you are prompted to continue the procedure, enter y when prompted. 14. Enter y when prompted regarding non-disruptive upgrade or replacement of the software. 15. Enter the path to the image.tgz file when you see the following prompt: What is the URL for the package? Restore the backup configuration on the target node by entering y and then entering the IP address of the server containing the varfs.tgz file when you see the following screen: 50 IBM System Storage: N6000 Series Hardware and Service Guide

77 **************************************************************** * Restore Backup Configuration * * This procedure only applies to storage controllers that * * are configured as an HA pair. * * * * Choose Yes to restore the "varfs" backup configuration * * from the SSH server. Refer to the Boot Device Replacement * * guide for more details. * * Choose No to skip the backup and recovery and return to the * * boot menu. * **************************************************************** Do you want to restore the backup configuration now? {yn} y Start the restore_backup procedure on this node s HA partner. The restore configuration has been copied from the HA partner to this node, would you like to use this restored copy now? {yn} 17. Copy the varfs.tgz file from the HA partner node to the target node by entering the following command from the partner node console: restore_backup target_node.netboot_ip.address Use the same IP address that was assigned to the target node and used for the netboot command. 18. Finish the restore backup on the target node by entering y when you see the following screen: The restore configuration has been copied from the HA partner to this node, would you like to use this restored copy now? {yn} y CF device: /dev/ad0 found... Checking /dev/ad0s1 file system... success. Mounting /dev/ad0s1 to /cfcard... success. Checking /cfcard/x86/freebsd mount point... success. Restoring backup configuration Reboot the target node by entering the following command on the target node console: boot_ontap 20. After the target node has rebooted and is displaying the Waiting for giveback message, complete the procedure by entering the following command on the partner node's console: cf giveback Installing the CompactFlash card and transferring system files disruptively using netboot You can transfer the system files and restore configuration information to your system by using netboot to copy the system files to the replacement CompactFlash card. Before you begin v You must have the blank replacement CompactFlash card you received from your provider. v You must have access to an HTTP server. v You must have access to the IBM N series support website at Chapter 4. Replacing N6000 series system devices 51

78 This enables you to download the necessary system files for your platform and version of Data ONTAP running on it. About this task You can use this disruptive procedure in a system that is running Data ONTAP 8.x. Attention: This procedure is disruptive, even when using it with a system in an HA configuration, because an immediate giveback and re-takeover occur during the procedure. If possible, you should use the nondisruptive procedure for HA configurations running Data ONTAP and later operating in 7-Mode. Procedure 1. Download and extract the file used for performing the netboot of your system: a. Download the appropriate netboot.tgz file for your platform from the IBM N series support website to a web-accessible directory. b. Change to the web-accessible directory. c. Extract the contents of the netboot.tgz file to the target directory by entering the following command: tar -zxvf netboot.tgz Your directory listing should contain the following directory: netboot/ 2. Download the image.tgz file from the IBM N series support website to the web-accessible directory. Your directory listing should contain the following file and directory: image.tgz netboot/ 3. If you are not already grounded, properly ground yourself. 4. Turn the controller module so that you can locate the CompactFlash card socket. 5. Align the CompactFlash card with the CompactFlash card socket or connector, and then firmly push the CompactFlash card into the socket or connector. The CompactFlash card socket is keyed. If you feel resistance while pushing the card into the socket, check the orientation of the card. 6. Check the CompactFlash card to make sure that it is seated squarely and completely in the socket or connector. If necessary, remove the CompactFlash card and reseat it into the socket. 7. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system. 8. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 9. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, push the cam handle to the closed position, and then tighten the thumbscrew. 10. The next step depends on your system configuration: If your system has... One controller module in the chassis Two controller modules in the chassis Then... Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. The node already began rebooting; go to the next step. 52 IBM System Storage: N6000 Series Hardware and Service Guide

79 11. Depending on your network configuration, enter one of the following commands at the LOADER prompt: If you... Have DHCP enabled Do not have DHCP enabled Then... Enter the following command: ifconfig e0a -auto Enter the following command: ifconfig e0a -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain filer_addr is the IP address of the storage system. netmask is the network mask of the storage system. gateway is the gateway for the storage system. dns_addr is the IP address of a name server on your network. dns_domain is the Domain Name System (DNS) domain name. If you use this optional parameter, you do not need a fully qualified domain name in the netboot server URL; you need only the server s host name. Note: To netboot the node when your system is running in 7-Mode Data ONTAP, use an IP address that is not the management IP address for the target. If your system is running clustered Data ONTAP, you can use the management IP address. Note: Other parameters might be necessary for your interface. For details, use the help ifconfig command at the LOADER prompt. 12. At the LOADER prompt, enter the following command: netboot The system begins to boot, but stops at the Boot menu. 13. Select the Install new software first option from the displayed menu. This menu option downloads and installs the new Data ONTAP image to the CompactFlash card. If you are prompted to continue the procedure, enter y when prompted. 14. The next step depends on which version of Data ONTAP is installed on your system: If your system is running... Data ONTAP 8.0 Then... Reboot the node by entering y when you see the following prompt: The node must be rebooted to start using the newly installed software. Do you want to reboot now? [y/n} y The controller module reboots, but stops at the Boot menu because the boot device was reformatted and the configuration data needs to be restored. Chapter 4. Replacing N6000 series system devices 53

80 If your system is running... Data ONTAP and later Then... Complete the following substeps: 1. Enter n to skip the backup recovery when you see the following prompt: ************************************************************** * Restore Backup Configuration * * This procedure only applies to storage controllers that * * are configured as an HA pair. * * * * Choose Yes to restore the varfs backup configuration * * from a TFTP server. Refer to the Boot Device Replacement * * guide for more details. * * Choose No to skip the back up recovery and return to the * * boot menu. * ************************************************************** Do you want to restore the backup configuration now? {yn} n Note: If your system is running Data ONTAP and later, the prompt says SSH server. 2. Reboot the node by entering y when you see the following prompt: The node must be rebooted to start using the newly installed software. Do you want to reboot now? {y/n} y The controller module reboots, but stops at the boot environment prompt. 3. Boot Data ONTAP by entering the following command at the boot environment prompt: boot_ontap The controller module reboots, but stops at the Boot menu because the boot device was reformatted and the configuration data needs to be restored. 15. Select the Update flash from backup config option from the displayed menu. 16. Enter y when you see the following prompt: This will replace all flash-based configuration with the last backup to disk. Are you sure you want to continue?: y The system automatically reboots when the update flash (syncflash) process is complete. 17. Your next step depends on your system configuration. If your system is... A stand-alone configuration Description You can begin using your system after the node reboots to complete restoring the backup configuration. 54 IBM System Storage: N6000 Series Hardware and Service Guide

81 If your system is... An HA pair Description Complete the following substeps after the replacement node is displaying the Waiting for Giveback message: 1. Enter the following command from the healthy node: cf giveback The replacement node takes back its storage, completes booting up, and then reboots and is again taken over by the healthy node. 2. Enter the following command from the healthy node: cf giveback The replacement node takes back its storage and completes booting up with the restored configuration. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Replacing a CompactFlash card in a system running Data ONTAP 8.2 (Cluster Mode) Replacing a CompactFlash card in clustered Data ONTAP involves shutting down the node, removing the old CompactFlash card, and transferring the system files to the new CompactFlash card using either a PC or laptop and card reader/writer, or netboot on a local server or netboot with an HTTP server. About this task This procedure is for systems running clustered Data ONTAP. If your system is running Data ONTAP 7-Mode, you must use that procedure instead. Shutting down a node operating in clustered Data ONTAP To shut down the impaired node you must determine the status of the node and, if necessary, take over the node so that the healthy node continues to serve data from the impaired node's storage. About this task Note: Leave the power supplies turned on at the end of this procedure to provide power to the healthy node. Procedure 1. If running clustered Data ONTAP, check the status of the nodes in the cluster: a. Enter the following command at the system console of either node: cluster show The command produces output similar to the following: Chapter 4. Replacing N6000 series system devices 55

82 Node Health Eligibility node1 true true node2 true true node3 true true node4 true true 4 entries were displayed. b. Take one of the following actions, depending on the result of the command: If... All nodes show true for both health and eligibility. The impaired node shows false for health. Any nodes show false for eligibility. Any nodes other than the impaired node show false for health. Then... Proceed to Step 3. Proceed to the next step. Resolve any cluster issues as needed before continuing with this procedure. Correct the problems that cause the health issues on the nodes before continuing with this procedure. 2. Either shut down or perform a takeover of the impaired node, depending on your configuration: If the impaired node is in... A stand-alone configuration and is running A stand-alone configuration and is not running An HA pair Then Enter the following command at the console: system node halt impaired_node_name 2. Go to step 5. Go to step Check the status of the impaired node by entering the following command from the healthy node's console: storage failover show 2. Take one of the following actions, depending on the result of the storage failover show command: v v v If the impaired node is not running or has been taken over by the healthy node, go to step 4. If the impaired node took over the healthy node, give back the healthy node using the storage failover giveback command from the impaired node, go to the next step. If the impaired node has not been taken over by the healthy node and is running, go to the next step. 3. Take over the impaired node by entering the following command: storage failover takeover -ofnode impaired_node_name When it is taken over, the impaired node automatically reboots and displays the Waiting for giveback... message. 56 IBM System Storage: N6000 Series Hardware and Service Guide

83 4. Wait at least two minutes after takeover of the impaired node to ensure that the takeover completed successfully. 5. With the impaired node showing the Waiting for giveback... message, shut it down. The method you use to shut down the node depends on whether remote management via a Remote LAN Module (RLM) is used. Is the RLM configured? Yes No Then... Log in to theimpaired node's RLM and issue the following command: system power off. At the impaired node's prompt, press Ctrl-C and respond Y to halt the node. 6. If in a stand-alone configuration, shut down the power supplies and unplug both power cords from the power source. Opening the system To access components inside the controller module, you must open the system. Procedure 1. If you are not already grounded, properly ground yourself. 2. Unplug the system cables and SFPs from the controller module, as needed, and keep track of where the cables were connected. Leave the cables in the cable clips of the cable management tray so that when you reinstall the cable management tray, the cables are already aligned. 3. Grasp the cable management tray by the side, gently push one side of the tray so that the arm slides across and off the retaining pin, and then lift it off the back of the controller module and set it aside. 4. Loosen the thumbscrew on the cam handle. 5. Pull the cam handle downward and slide the controller module out of the system until it catches. 6. Push in the release latch on the left side of the controller module and slide the controller module out of the system and set it on a stable, antistatic surface. Make sure that you support the bottom of the controller module with your free hand. Removing the CompactFlash card from the controller module You must remove the old or failed CompactFlash card from the target controller module in your system prior to installing the new or replacement CompactFlash card. Procedure 1. If you are not already grounded, properly ground yourself. 2. Remove any cabling from the controller module, if necessary. Make sure that you keep track of where the cables were connected into the controller module. Chapter 4. Replacing N6000 series system devices 57

84 3. Remove the controller module and turn it so that you can see where the CompactFlash card is located Rear of the N6040, N6060, or N6070 controller module CompactFlash card 4. Grasp the edges of the CompactFlash card and gently lift it out of the socket. 5. Set the CompactFlash card aside. 6. Determine which method you will use to transfer the system files to the replacement boot device; using a PC or laptop with a card reader/writer, or using netboot. Installing the CompactFlash card After you remove the old CompactFlash card, you must transfer system files and restore configuration information to your replacement CompactFlash card using one of several methods. Before you begin Note: For systems running 7-Mode, IBM recommends using the nondisruptive procedure for HA pair running Data ONTAP and later. For systems running clustered Data ONTAP, IBM recommends using the nondisruptive procedure for HA pair running Data ONTAP 8.2 and later. Installing the CompactFlash card using a PC or laptop with a card reader/writer You can transfer the system files to your replacement CompactFlash card using a PC or laptop with a CompactFlash card reader/writer prior to installing the replacement boot device into your controller. 58 IBM System Storage: N6000 Series Hardware and Service Guide

85 Before you begin For copying system files to the CompactFlash card, you must have the following: v The blank replacement CompactFlash card you received from your provider v A PC or laptop with a CompactFlash reader/writer that is running Windows XP or later and that has a.zip program, such as WinZip, installed v Access to the IBM N series support website. This enables you to download the necessary system files for your platform and version of Data ONTAP running on it. Transferring the system files to a CompactFlash card in a system running clustered Data ONTAP using a PC or laptop: You can transfer system files to the CompactFlash card in a clustered Data ONTAP system to the replacement boot device using a PC or laptop that has a CompactFlash card reader/writer. Procedure 1. Download the *.tgz system image file from the IBM N series support website to a temporary folder on your PC. The file you download is named <release_name>_sysfiles_q.tgz. 2. Insert the blank CompactFlash card into the CompactFlash card reader/writer and then access the boot device using Windows Explorer. Note: The CompactFlash card is pre-formatted. If you reformat the card, it should be in FAT-16 format only. 3. Create a folder in the root partition of the CompactFlash card, naming it x86_ Create a freebsd/ subfolder, and then create image1/ and image2/ subfolders inside freebsd/. The directory structure resembles the following example: x86_64/ freebsd/ image1/ image2/ 5. Navigate to the folder containing the *.tgz file on the PC and click the *.tgz file. 6. Extract the files to the following locations on the CompactFlash card: v freebsd/image1 v freebsd/image2 7. Return to the open zip-file window that displays the contents of the *.tgz file you want to install. 8. Uncompress and untar the contents of the file named diags.tgz to the root directory on the CompactFlash card. This creates the directory structure for the diagnostics utilities and firmware. 9. Verify the directory structure on the CompactFlash card. It should resemble the following example: Chapter 4. Replacing N6000 series system devices 59

86 x86_64/ freebsd/ image1/ image2/ diag/ firmware/ common/ firmware/ 10. Close the.zip program. 11. Right-click the drive letter that is mapped to the CompactFlash card, select Eject to remove the CompactFlash card from the PC or laptop, and then remove the CompactFlash card from the CompactFlash card reader/writer. Installing the CompactFlash card into a controller running clustered Data ONTAP: After the system files are transferred to the replacement boot device, you must install it into the controller. Procedure 1. If you are not already grounded, properly ground yourself. 2. Turn the controller module so that you can locate the CompactFlash card socket. 3. Align the CompactFlash card with the CompactFlash card socket or connector, and then firmly push the CompactFlash card into the socket or connector. The CompactFlash card socket is keyed. If you feel resistance while pushing the card into the socket, check the orientation of the card. 4. Check the CompactFlash card to make sure that it is seated squarely and completely in the socket or connector. If necessary, remove the CompactFlash card and reseat it into the socket. 5. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system. 6. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 7. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, push the cam handle to the closed position, and then tighten the thumbscrew. 8. The next step depends on your system configuration: If your system has... One controller module in the chassis Two controller modules in the chassis Then... Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. The node already began rebooting; go to the next step. Rebooting the controller running clustered Data ONTAP: After copying the system files to the new CompactFlash card and installing the card in the controller, you must reboot the node. 60 IBM System Storage: N6000 Series Hardware and Service Guide

87 Procedure 1. Power on the node, if necessary, and log in to its console. 2. Reboot the system and interrupt the boot process by pressing Ctrl-C when you see the following message on the console: Starting AUTOBOOT press ctrl-c to abort 3. Set the following variable to true: setenv bootarg.init.boot_clustered true 4. Boot Data ONTAP by entering the following command at the prompt: autoboot 5. Press Ctrl-C during the boot process to access the boot menu. 6. Log in to the failover partner of the node being serviced and enter the following command: storage failover giveback -fromnode node_name -require-partner-waiting false node_name is the name of the node currently holding the storage of the node whose CompactFlash card you replaced. 7. Enter y when the following message is displayed: WARNING Do you wish to initiate giveback when the partner may not be waiting for giveback state (y or n): y 8. Select the Update flash from backup config option from the displayed menu. 9. If the node's disks are not automatically returned to its failover partner, contact technical support for help. Installing the CompactFlash card and transferring system files nondisruptively using netboot in systems running Data ONTAP and later You can transfer the system files and restore configuration information to your HA pair nondisruptively by using netboot and the partner node. Before you begin v You must have the blank replacement CompactFlash card you received from your provider. v You must have a network interface to the HA pair partner node. v You must have a separate IP address to use to netboot the target node. About this task The following procedure is written with the assumption that the target node has been taken over by the partner node and that the target node has booted to the boot environment prompt after the takeover has been completed. Attention: You can only use this nondisruptive procedure in an HA pair that is running Data ONTAP and later. Procedure 1. Download and extract the file used for performing the netboot of your system: a. Download the appropriate netboot.tgz file for your platform from the IBM N series support website to a web-accessible directory. b. Change to the web-accessible directory. Chapter 4. Replacing N6000 series system devices 61

88 c. Extract the contents of the netboot.tgz file to the target directory by entering the following command: tar -zxvf netboot.tgz Your directory listing should contain the following directory: netboot/ 2. Download the image.tgz file from the IBM N series support website to the web-accessible directory. Your directory listing should contain the following file and directory: image.tgz netboot/ 3. If you are not already grounded, properly ground yourself. 4. Turn the controller module so that you can locate the CompactFlash card socket. 5. Align the CompactFlash card with the CompactFlash card socket or connector, and then firmly push the CompactFlash card into the socket or connector. The CompactFlash card socket is keyed. If you feel resistance while pushing the card into the socket, check the orientation of the card. 6. Check the CompactFlash card to make sure that it is seated squarely and completely in the socket or connector. If necessary, remove the CompactFlash card and reseat it into the socket. 7. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system. 8. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 9. Reinstall the controller module: If your system is in... An HA pair Then perform these steps Be prepared to interrupt the boot process. The controller module begins to boot as soon as it is fully seated in the chassis. 2. With the cam handle in the open position, insert the controller module into the chassis, firmly pushing until the controller module meets the midplane, and then close the cam handle so that the latch clicks into the locked position and the controller module is fully seated. Attention: Do not use excessive force when sliding the controller module into the chassis; you might damage the connectors. 3. As the system begins to boot, press Ctrl-C to interrupt the boot process when you see the message Press Ctrl-C for Boot Menu. 4. From the boot menu, select the option for Maintenance mode. 62 IBM System Storage: N6000 Series Hardware and Service Guide

89 If your system is in... A stand-alone configuration Then perform these steps With the cam handle in the open position, insert the controller module motherboard tray into the chassis, firmly pushing until the controller module meets the midplane, and then close the cam handle so that the latch clicks into the locked position and the controller module is fully seated. Attention: Do not use excessive force when sliding the controller module into the chassis; you might damage the connectors. 2. Reconnect the power cables to the power supplies and to the power sources, turn on the power to start the boot process, and then press Ctrl-C to interrupt the boot process when you see the message Press Ctrl-C for Boot Menu. 3. From the boot menu, select the option for Maintenance mode. Important: During the boot process, you might see the following prompts: v A prompt warning of a system ID mismatch and asking to override the system ID. v A prompt warning that when entering Maintenance mode in a HA configuration you must ensure that the partner remains down. You can safely respond Y to these prompts. 10. If the boot process was interrupted too late and the Boot Menu appeared, perform the following steps: a. Select the Maintenance mode option from the displayed menu. b. After the system boots to Maintenance mode, enter the following command at the prompt: halt After you issue the command, wait until the system stops at the LOADER prompt. Important: During the boot process, you might see the following prompts: v A prompt warning of a system ID mismatch and asking to override the system ID. v A prompt warning that when entering Maintenance mode in a HA configuration you must ensure that the partner remains down. You can safely respond y to these prompts. 11. Set the following boot environment variable at the LOADER prompt on the target node console: setenv bootarg.init.boot_clustered true 12. Depending on your network configuration, enter one of the following commands at the LOADER prompt: Chapter 4. Replacing N6000 series system devices 63

90 If you... Have DHCP enabled Do not have DHCP enabled Then... Enter the following command: ifconfig e0a -auto Enter the following command: ifconfig e0a -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain filer_addr is the IP address of the storage system. netmask is the network mask of the storage system. gateway is the gateway for the storage system. dns_addr is the IP address of a name server on your network. dns_domain is the Domain Name System (DNS) domain name. If you use this optional parameter, you do not need a fully qualified domain name in the netboot server URL; you need only the server s host name. Note: To netboot the node when your system is running in 7-Mode Data ONTAP, use an IP address that is not the management IP address for the target. If your system is running clustered Data ONTAP, you can use the management IP address. Note: Other parameters might be necessary for your interface. For details, use the help ifconfig command at the LOADER prompt. 13. At the boot environment prompt, enter the following command: netboot Select the Install new software first option from the displayed menu. This menu option downloads and installs the new Data ONTAP image to the CompactFlash card. If you are prompted to continue the procedure, enter y when prompted. 15. Enter y when prompted regarding non-disruptive upgrade or replacement of the software. 16. Enter the path to the image.tgz file when you see the following prompt: What is the URL for the package? Restore the backup configuration by entering y when you see the following screen: 64 IBM System Storage: N6000 Series Hardware and Service Guide

91 **************************************************************** * Restore Backup Configuration * * This procedure only applies to storage controllers that * * are configured as an HA pair. * * * * Choose Yes to restore the "varfs" backup configuration * * from the SSH server. Refer to the Boot Device Replacement * * guide for more details. * * Choose No to skip the backup recovery and return to the * * boot menu. * **************************************************************** Do you want to restore the backup configuration now? {yn} y Start the restore_backup procedure on this node s HA partner. The restore configuration has been copied from the HA partner to this node. Would you like to use this restored copy now? {yn} 18. Copy the varfs.tgz file from the HA partner node to the target node by entering the following command from the partner node console: system node run -node partner_node_name -command restore_backup target_node.netboot_ip.address Use the same IP address that was used for the netboot command. 19. Finish the restore backup procedure on the target node by entering y when you see the following screen: The restore configuration has been copied from the HA partner to this node, would you like to use this restored copy now? {yn} y CF device: /dev/ad0 found... Checking /dev/ad0s1 file system... success. Mounting /dev/ad0s1 to /cfcard... success. Checking /cfcard/x86/freebsd mount point... success. Restoring backup configuration Reboot the target node by entering the following command on the target node console: system node reboot -node node-name 21. After the target node has rebooted, complete the procedure by entering the following command on the partner node console: storage failover giveback -ofnode partner_node_name If the giveback is vetoed, you can consider forcing giveback by issuing the following command: storage failover giveback -ofnode partner_node_name -require-partnerwaiting true -override-vetoes true If you are using Data ONTAP 8.2, see the Clustered Data ONTAP High-Availability Configuration Guide on the IBM N series support website for information on giveback vetoes. Installing the CompactFlash card and transferring system files disruptively using netboot You can transfer the system files and restore configuration information to your HA pair disruptively by using netboot to copy the system files to the replacement CompactFlash card. Chapter 4. Replacing N6000 series system devices 65

92 Before you begin v v v You must have the blank replacement CompactFlash card that you received from your provider. You must have access to an HTTP server. You must have access to the IBM N series support website at This enables you to download the necessary system files for your platform and version of Data ONTAP running on it. About this task You can use this disruptive procedure only in a system that is running clustered Data ONTAP 8.x. Note: This procedure is disruptive, even when using it with a system in an HA pair, because an immediate giveback and re-takeover occur during the procedure. Procedure 1. Download and extract the file used for performing the netboot of your system: a. Download the appropriate netboot.tgz file for your platform from the IBM N series support website to a web-accessible directory. b. Change to the web-accessible directory. c. Extract the contents of the netboot.tgz file to the target directory by entering the following command: tar -zxvf netboot.tgz Your directory listing should contain the following directory: netboot/ 2. Download the image.tgz file from the IBM N series support website to the web-accessible directory. Your directory listing should contain the following file and directory: image.tgz netboot/ 3. If you are not already grounded, properly ground yourself. 4. Turn the controller module so that you can locate the CompactFlash card socket. 5. Align the CompactFlash card with the CompactFlash card socket or connector, and then firmly push the CompactFlash card into the socket or connector. The CompactFlash card socket is keyed. If you feel resistance while pushing the card into the socket, check the orientation of the card. 6. Check the CompactFlash card to make sure that it is seated squarely and completely in the socket or connector. If necessary, remove the CompactFlash card and reseat it into the socket. 7. Align the end of the controller module with the opening in the chassis, and then gently push the controller module halfway into the system. 8. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 9. Reinstall the controller module: 66 IBM System Storage: N6000 Series Hardware and Service Guide

93 If your system is in... An HA pair A stand-alone configuration Then perform these steps Be prepared to interrupt the boot process. The controller module begins to boot as soon as it is fully seated in the chassis. 2. With the cam handle in the open position, insert the controller module into the chassis, firmly pushing until the controller module meets the midplane, and then close the cam handle so that the latch clicks into the locked position and the controller module is fully seated. Attention: Do not use excessive force when sliding the controller module into the chassis; you might damage the connectors. 3. As the system begins to boot, press Ctrl-C to interrupt the boot process when you see the message Press Ctrl-C for Boot Menu. 4. From the boot menu, select the option for Maintenance mode. 1. With the cam handle in the open position, insert the controller module motherboard tray into the chassis, firmly pushing until the controller module meets the midplane, and then close the cam handle so that the latch clicks into the locked position and the controller module is fully seated. Attention: Do not use excessive force when sliding the controller module into the chassis; you might damage the connectors. 2. Reconnect the power cables to the power supplies and to the power sources, turn on the power to start the boot process, and then press Ctrl-C to interrupt the boot process when you see the message Press Ctrl-C for Boot Menu. 3. From the boot menu, select the option for Maintenance mode. Important: During the boot process, you might see the following prompts: v A prompt warning of a system ID mismatch and asking to override the system ID. v A prompt warning that when entering Maintenance mode in a HA configuration you must ensure that the partner remains down. You can safely respond Y to these prompts. 10. If the boot process was interrupted too late and the Boot Menu appeared, perform the following steps: a. Select the Maintenance mode option from the displayed menu. Chapter 4. Replacing N6000 series system devices 67

94 b. After the system boots to Maintenance mode, enter the following command at the prompt: halt After you issue the command, wait until the system stops at the LOADER prompt. Important: During the boot process, you might see the following prompts: v A prompt warning of a system ID mismatch and asking to override the system ID. v A prompt warning that when entering Maintenance mode in a HA configuration you must ensure that the partner remains down. You can safely respond y to these prompts. 11. Set the following boot environment variables at the boot environment prompt on the target node console: setenv bootarg.init.boot_clustered true setenv bootarg.init.usebootp false Note: The bootarg.init.usebootp variable needs to be set only if your system is running Data ONTAP Depending on your network configuration, enter one of the following commands at the LOADER prompt: If you... Have DHCP enabled Do not have DHCP enabled Then... Enter the following command: ifconfig e0a -auto Enter the following command: ifconfig e0a -addr=filer_addr -mask=netmask -gw=gateway -dns=dns_addr -domain=dns_domain filer_addr is the IP address of the storage system. netmask is the network mask of the storage system. gateway is the gateway for the storage system. dns_addr is the IP address of a name server on your network. dns_domain is the Domain Name System (DNS) domain name. If you use this optional parameter, you do not need a fully qualified domain name in the netboot server URL; you need only the server s host name. Note: To netboot the node when your system is running in 7-Mode Data ONTAP, use an IP address that is not the management IP address for the target. If your system is running clustered Data ONTAP, you can use the management IP address. Note: Other parameters might be necessary for your interface. For details, use the help ifconfig command at the LOADER prompt. 68 IBM System Storage: N6000 Series Hardware and Service Guide

95 13. At the LOADER prompt, enter the following command: netboot The system begins to boot, but stops at the Boot menu. 14. Select the Install new software first option from the displayed menu. This menu option downloads and installs the new Data ONTAP image to the CompactFlash card. If you are prompted to continue the procedure, enter y when prompted. 15. Your next step depends on which version of Data ONTAP is installed on your system. If your system is running... Data ONTAP 8.0 Data ONTAP and later Then... Reboot the node by entering y when you see the following prompt: The node must be rebooted to start using the newly installed software. Do you want to reboot now? [y/n} y The controller module reboots, but stops at the Boot menu because the boot device was reformatted and the configuration data needs to be restored. Complete the following substeps: 1. Enter n to skip the backup recovery when you see the following prompt: ************************************************************** * Restore Backup Configuration * * This procedure only applies to storage controllers that * * are configured as an HA pair. * * * * Choose Yes to restore the varfs backup configuration * * from a TFTP server. Refer to the Boot Device Replacement * * guide for more details. * * Choose No to skip the back up recovery and return to the * * boot menu. * ************************************************************** Do you want to restore the backup configuration now? {yn} n Note: If your system is running Data ONTAP and later, the prompt says SSH server. 2. Reboot the node by entering y when you see the following prompt: The node must be rebooted to start using the newly installed software. Do you want to reboot now? {y/n} y The controller module reboots, but stops at the boot environment prompt. 3. Boot Data ONTAP by entering the following command at the boot environment prompt: boot_ontap The controller module reboots, but stops at the Boot menu because the boot device was reformatted and the configuration data needs to be restored. 16. Select the Update flash from backup config option from the displayed menu. If you are asked to continue with the update, enter y when prompted. 17. Your next step depends on your system configuration. Chapter 4. Replacing N6000 series system devices 69

96 If your system is... A stand-alone configuration An HA pair Description You can begin using your system after the node reboots to complete restoring the backup configuration. Complete the following substeps after the replacement node is displaying the Waiting for Giveback message: 1. Enter the following command from the healthy node: v In Data ONTAP or earlier: storage failover giveback -fromnode partner_node_name v In Data ONTAP or later: storage failover giveback -ofnode partner_node_name The replacement node takes back its storage, completes booting up, and then reboots and is again taken over by the healthy node. If the giveback is vetoed, you can consider forcing giveback by issuing the following command: storage failover giveback -ofnode partner_node_name -require-partnerwaiting true -override-vetoes true If you are using Data ONTAP 8.2, see the Clustered Data ONTAP High-Availability Configuration Guide on the IBM N series support website for information on giveback vetoes. 2. Enter the following command from the healthy node: v In Data ONTAP or earlier: storage failover giveback -fromnode healthy_node_name v In Data ONTAP or later: storage failover giveback -ofnode replacement_node_name The replacement node takes back its storage and completes booting up with the restored configuration. 3. Enter the storage failover show-giveback command to monitor the progress of the giveback operation. 4. After the giveback operation is complete, enter the storage failover show command to confirm that the HA pair is healthy and takeover is possible. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. 70 IBM System Storage: N6000 Series Hardware and Service Guide

97 Replacing a fan module in a system To replace a fan module, you must perform the following tasks in order. About this task The fan module is hot-swappable and can be replaced without interrupting service. Removing a fan module You can remove a fan module without powering down the system. About this task Attention: You must replace the fan module within two minutes of removing it from the chassis. System airflow is disrupted and the controller module shuts down after two minutes to avoid overheating. In high-availability configurations, a takeover occurs. Procedure 1. If you are not already grounded, properly ground yourself. 2. Remove the bezel (if necessary) with two hands, by grasping the openings on each side of the bezel, and then pulling it toward you until the bezel releases from the four ball studs on the chassis frame. 3. Identify the fan module that you must replace. Check the console error messages and look at the Status LED on each fan module cam handle. A failed fan module LED is solid amber, indicating that the fan module failed or it is not receiving power. 4. Press down the release latch on the fan module cam handle and pull the cam handle downward. The fan module moves a little bit away from the chassis. Chapter 4. Replacing N6000 series system devices 71

98 Figure 12. Replacing a fan module Cam handle Fan module Cam handle release latch Fan module status LED 5. Pull the fan module straight out from the chassis, making sure that you support it with your free hand, so that it does not swing out of the chassis. Attention: The fan modules are short. Always support the bottom of the module with your free hand so that it does not suddenly drop free from the chassis and injure you. 6. Set the fan module aside. Installing a fan module To replace the fan module, you must perform the following steps in order. Procedure 1. If you are not already grounded, properly ground yourself. 2. Remove the bezel, if necessary. 3. Insert the replacement fan module into the chassis by aligning it with the opening and sliding it into the chassis. 4. Push firmly on the fan module housing to ensure that it is seated all the way into the chassis. 72 IBM System Storage: N6000 Series Hardware and Service Guide

99 5. Swing the cam handle up to its closed position, making sure that the cam handle release latch clicks into the locked position. 6. Reinstall the bezel. 7. Run diagnostics on the replacement fan module during the next scheduled system downtime. See the IBM System Storage N series Diagnostics Guide for more information. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Installing or replacing the RLM in a system To install or replace a Remote LAN Module (RLM) in the system, you must perform the following tasks in order. About this task v You can use this procedure with all versions of Data ONTAP supported by your system. v This procedure refers to HA pairs, which in releases prior to Data ONTAP 8.0 were called active/active configurations. v All other components in the system must be functioning properly; if not, contact technical support. Minimum required RLM firmware version The N6040, N6060, or N6070 storage systems require an RLM card with firmware version 2.0 or later. This procedure includes the verification of the firmware version on your new RLM card and directs you to instructions for downloading updated firmware, if needed. Shutting down the node You shut down a node using different procedures, depending on whether it is a stand-alone system or part of an HA pair. Shutting down a node in an HA pair To shut down the node you must determine the status of the node and, if necessary, take over the node so that the partner continues to serve data from the node's storage. About this task Note: Leave the power supplies turned on at the end of this procedure to provide power to the partner node. Procedure 1. If running clustered Data ONTAP, check the status of the nodes in the cluster: a. Enter the following command at the system console of either node: cluster show The command produces output similar to the following: Chapter 4. Replacing N6000 series system devices 73

100 Node Health Eligibility node1 true true node2 true true node3 true true node4 true true 4 entries were displayed. b. Take one of the following actions, depending on the result of the command: If... All nodes show true for both health and eligibility. The impaired node shows false for health. Any nodes show false for eligibility. Any nodes other than the impaired node show false for health. Then... Proceed to Step 3. Proceed to the next step. Resolve any cluster issues as needed before continuing with this procedure. Correct the problems that cause the health issues on the nodes before continuing with this procedure. 2. Check the status of the target node (the node you want to perform maintenance on) by entering the following command at the system console of either node: If your system is configured in... 7-Mode Cluster-Mode Then issue this command... cf status storage failover show 3. Take one of the following actions, depending on the result of the cf status or storage failover show command: If... Neither node is in takeover mode The partner node took over the target node The target node took over the partner node Then... Go to the next step in this procedure. The target node is in a state where you can begin removing it from the system chassis. 1. Correct the problem that caused the takeover. 2. Run the cf giveback command (if in a 7-Mode system) or storage failover giveback (if in a Cluster-Mode system) from the target node console. 3. Go back to the beginning of this procedure. 4. Take over and power down the impaired node by taking the following steps: a. Enter one of the following commands from the healthy node s console and wait for the takeover to complete: For systems operating in... 7-Mode Clustered Data ONTAP Issue the command... cf takeover In Data ONTAP 8.2 or later: storage failover takeover -ofnode impaired_node_name 74 IBM System Storage: N6000 Series Hardware and Service Guide

101 The impaired node is taken over and then automatically reboots and displays the Waiting for giveback... message. Note: Leave the power supplies turned on to provide power to the healthy node. b. Wait at least two minutes after takeover of the impaired node to ensure that the takeover completed successfully. c. With the impaired node showing the Waiting for giveback... message, shut it down. The method you use to shut down the node depends on whether remote management via a Remote LAN Module (RLM) is used. Is the RLM configured? Yes No Then... Log in to the impaired node's RLM and issue the following command: system power off At the impaired node's prompt, press Ctrl-C and respond Y to halt the node. Shutting down the node in a stand-alone system For a node that is not configured with a high-availability (HA) partner, you must perform a clean shutdown (ensuring that all data has been written to disk) and disconnect the power supplies. Procedure 1. Enter one of the following commands from the system console: If your system is configured as a... 7-Mode system Cluster-Mode Then issue this command... halt halt local Attention: You must perform a clean system shutdown before replacing system components to avoid losing unwritten data in the NVRAM. The NVRAM adapter has a red LED that blinks if there is unwritten data in the NVRAM. If this LED is flashing red after you enter the halt command, reboot your system and try halting it again. 2. If you are not already grounded, properly ground yourself. 3. Turn off the power supplies, unplug both power cords from the power source, and then remove the power cords. If your system uses... AC power supplies DC power supplies Then... Unplug the power cords from the power source, and then remove the power cords. Remove the power at the DC source, and remove the DC wires, if necessary. Opening the system To access components inside the controller module, you must open the system. Procedure 1. If you are not already grounded, properly ground yourself. Chapter 4. Replacing N6000 series system devices 75

102 2. Unplug the system cables from the controller module, as needed, and keep track of where the cables were connected. Leave the cables in the cable clips of the cable management tray so that when you reinstall the cable management tray, the cables are already aligned. 3. Grasp the cable management tray by the blue touch points on the arms, gently push one side of the tray so that the arm slides off the retaining pin, and then lift the tray off the back of the controller module and set it aside. 4. Loosen the thumbscrew on the cam handle. 5. Pull the cam handle downward and slide the controller module out of the system until it catches. 6. Push in the blue release latch on the left side of the controller module and slide the controller module out of the system. Release the latch after you can slide the controller module. Make sure that you support the bottom of the controller module with your free hand. Removing the RLM To remove the RLM from the system, you must perform a specific sequence of steps. Procedure 1. Locate the RLM in the center of the riser housing, using the RLM label on the controller module DIMM cover as a reference. 2. Pinch the retaining tabs on the RLM, and then lift the RLM straight out of the socket and the housing. 3. Set the RLM aside. Installing the RLM You must complete a specific set of steps to install the RLM in the system. Procedure Align the edges of the RLM with the guide slots on the edges of the riser housing, lower the RLM into place, and then press the RLM into the socket by using your thumbs in the top center of the RLM. Make sure that you press down at the places indicated by the sticker on the RLM housing. Reinstalling the controller module and booting the system After you replace a component within the controller module, you must reinstall the controller module in the system chassis. About this task Attention: For HA pairs, the sequence in which you reinstall the controller module is especially important because it attempts to reboot as soon as you completely seat it in the chassis. Procedure 1. Close the DIMM cover and side panel, if necessary. 2. Align the end of the controller module with the opening in the chassis, if necessary, and then gently push the controller module halfway into the system. 3. Reinstall the cable management tray and recable the system, as needed. 76 IBM System Storage: N6000 Series Hardware and Service Guide

103 When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 4. Push the controller module all the way into the chassis. If your system is in... An HA pair A stand-alone configuration Then... The node reboots as soon as you seat the controller module completely into the chassis. Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. 5. Firmly push the cam handle to finish seating the controller module in the system, and then push the cam handle to the closed position. 6. Tighten the thumbscrew beneath the cam handle. 7. Check the status and firmware level of the RLM by entering the following command at the system prompt: rlm status 8. Check the current RLM firmware level posted on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Refer to the list of current RLM firmware versions in the System, RLM, SP, and Disk Shelf Firmware Matrix. If your RLM firmware version is... Earlier than the latest RLM firmware version posted on the IBM N series support website The latest RLM firmware version posted on the IBM N series support website Then... Download and install the latest version of firmware from the website, following the posted installation instructions for that level of firmware. Go to the next step. Connecting the RLM You must use this procedure to connect the RLM and make sure it is running with firmware version 2.0 or later. Procedure 1. Plug one end of an RJ-45 cable into the RLM port, and then plug the other end into a 10/100 Ethernet network port. The network for the RLM should be a 10/100 Ethernet network, or a network that can autonegotiate settings to a 10/100 Ethernet network. 2. Check the link LED on the RLM port to ensure that it is green and that the link is active. 3. Proceed to setup or test the RLM: If you are... Installing an RLM for the first time Replacing an existing RLM Then proceed to... Set up and then test the RLM Test the RLM Setting up a new RLM When installing an RLM in the system for the first time (rather than replacing an existing RLM), you must run the rlm setup command and respond to its prompts to configure the RLM. Chapter 4. Replacing N6000 series system devices 77

104 Before you begin The RLM requires AutoSupport to be activated and configured. If AutoSupport is not set up and configured, do so at this time. For more information, see the Data ONTAP System Administration Guide. This guide, and other Data ONTAP documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Procedure 1. Enter the following command from the system console to begin the RLM setup script: rlm setup Note: Your screen output might be different from this example, depending on which version of Data ONTAP is running on your system. mysystem-1> rlm setup The Remote LAN Module (RLM) provides remote management capabilities including console redirection, logging and power control. It also extends AutoSupport by sending additional system event alerts. Your AutoSupport settings are used for sending these alerts via over the RLM LAN interface. Would you like to configure the RLM LAN interface [y]: y Would you like to enable DHCP on the RLM LAN interface [y]: n Please enter the IP address for the RLM []: Please enter the netmask for the RLM []: Please enter the IP address for the RLM gateway []: mysystem-1> 2. Check the status and settings of the RLM by entering the following command from the system console: rlm status mysystem-1> rlm status Remote LAN Manager Part Number: Revision: 32 Serial Number: 2P6256 Firmware Version: 2.0 Fri May 30 16:27:58 PST 2008 Mgmt MAC Address: 00:A0:98:01:9A:F8 Using DHCP no IP Address: Netmask: Gateway: Check the current RLM firmware level posted on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Refer to the list of current RLM firmware versions in the System, RLM, SP, and Disk Shelf Firmware Matrix. If your RLM firmware version is... Earlier than the latest RLM firmware version posted on the IBM N series support website The latest RLM firmware version posted on the IBM N series support website Then... Download and install the latest version of firmware from the website, following the posted installation instructions for that level of firmware. Go to the next step. 78 IBM System Storage: N6000 Series Hardware and Service Guide

105 4. Test the RLM AutoSupport functions by sending a test to AutoSupport with the following command from the system console: rlm test autosupport Verifying the firmware version and testing an RLM You must connect the RLM, check to ensure it has firmware version 2.0 or later, and then use the RLM's command-line interface to test its features. About this task You must log in to the RLM over its network interface by using a secure shell client application, such as SSH, OpenSSH, or PuTTY. Insecure connection protocol applications, such as Telnet, are not available. For more information about secure shell client applications, see the Data ONTAP System Administration Guide. This document, as well as other Data ONTAP documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Procedure 1. Log into the RLM console from a remote host acting as a system console by entering the following command from the administration host prompt: ssh user_name@rlm_ip_address Note: Root login is not allowed for the RLM. If you need to log in with root credentials, use naroot for root login. admin-host> ssh admin@ admin@ s password:********* RLM admin-host> 2. Verify the version of RLM firmware by entering the following command in administration mode: version 3. Update the firmware if necessary: If the RLM firmware version is... Then or later Proceed to the next step in this procedure. Earlier than 2.0 You must update the firmware. For instructions on downloading firmware for your RLM, see the Data ONTAP Upgrade Guide on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. 4. Check the RLM status by entering the following command from the RLM console: rlm status 5. Exit the RLM console by entering the following command: exit 6. Connect to the system console using the serial connection or through telnet. 7. Test the RLM AutoSupport functions by sending a test to AutoSupport with the following command from the system console: rlm test autosupport Running diagnostics on the RLM You should run diagnostics on the RLM to confirm its operation. Chapter 4. Replacing N6000 series system devices 79

106 Procedure 1. Reboot the system and interrupt the boot process. 2. Enter the following at the prompt: boot_diags 3. Select the Agent and RLM Diagnostic option from the main menu, and then run the Comprehensive Test. 4. Correct any errors. 5. Exit diagnostics by entering the following command: exit 6. Return the system to normal operation: If your system is... In an HA pair Stand-alone Then issue this command Run the cf giveback command (if in a 7-Mode system) or storage failover giveback (if in a Cluster-Mode system) from the partner node console.from the partner node console. 2. On the console of the target controller module, select option 1 from the displayed menu to boot Data ONTAP: Normal Boot (option 1) Enter the following command at the prompt to boot the system: boot_ontap Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Replacing a power supply on a system To replace a power supply in the system, you must perform a number of tasks in order. About this task v All other components in the system must be functioning properly; if not, contact technical support. v The power supplies are redundant, and are hot-swappable and interchangeable. v To prevent system downtime, replace only one power supply at a time. Removing a power supply To remove a power supply, you must follow the steps to identify the one you want to remove, disconnect the power cord, and remove the power supply from the chassis. About this task Attention: You should replace the power supply within 60 minutes of removing it from the chassis. Procedure 1. Identify the power supply you want to replace, based on console error messages or through the LEDs on the power supplies. 2. If you are not already grounded, properly ground yourself. 80 IBM System Storage: N6000 Series Hardware and Service Guide

107 3. Turn off the target power supply and remove the power cord by completing the following substeps, using the illustration for reference: Figure 13. Replacing a power supply Power supply Cam handle release latch LEDs Cam handle Power cord locking mechanism a. Pinch the tab on the locking mechanism of the cable retainer clip, and then open the retainer clip. b. Slide the retainer clip off the cable. c. Unplug the power cord from the power supply and the power source, and then remove the power cord 4. Press down the release latch on the power supply cam handle, lower the cam handle to the fully open position, and then slide the power supply out of the chassis. Make sure that you support the power supply with your free hand. Installing a power supply To install a power supply, you must follow these steps to place the power supply in the chassis, secure it, and reconnect the power cord. Chapter 4. Replacing N6000 series system devices 81

108 Procedure 1. Make sure that the on/off switch of the new power supply is in the Off position. 2. If you are not already grounded, properly ground yourself. 3. Align the edges of the power supply with the opening in the system chassis and gently push the power supply into the chassis until it is almost flush with the back of the system. 4. Push on the edges of the power supply to seat it all the way into the chassis, and then push the cam handle to the closed position, making sure that the cam handle release latch clicks into its locked position. 5. Reconnect the power cord and secure it to the power supply using the cable retaining clip. 6. Reconnect the power supply to the power source, and then turn it on. The power supply should run with no fault LEDs lit. 7. Run diagnostics on the power supply at next reboot. See the IBM System Storage N series Diagnostics Guide for more information. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Replacing the Real-Time Clock (RTC) battery in a system To replace the RTC battery in the controller module, you must perform the following tasks in order. About this task v You can use this procedure with all versions of Data ONTAP supported by your system. v This procedure refers to HA pairs, which in releases prior to Data ONTAP 8.0 were called active/active configurations. v All other components in the system must be functioning properly; if not, contact technical support. Shutting down the node You shut down a node using different procedures, depending on whether it is a stand-alone system or part of an HA pair. Shutting down a node in an HA pair To shut down the node you must determine the status of the node and, if necessary, take over the node so that the partner continues to serve data from the node's storage. About this task Note: Leave the power supplies turned on at the end of this procedure to provide power to the partner node. Procedure 1. If running clustered Data ONTAP, check the status of the nodes in the cluster: a. Enter the following command at the system console of either node: cluster show 82 IBM System Storage: N6000 Series Hardware and Service Guide

109 The command produces output similar to the following: Node Health Eligibility node1 true true node2 true true node3 true true node4 true true 4 entries were displayed. b. Take one of the following actions, depending on the result of the command: If... All nodes show true for both health and eligibility. The impaired node shows false for health. Any nodes show false for eligibility. Any nodes other than the impaired node show false for health. Then... Proceed to Step 3. Proceed to the next step. Resolve any cluster issues as needed before continuing with this procedure. Correct the problems that cause the health issues on the nodes before continuing with this procedure. 2. Check the status of the target node (the node you want to perform maintenance on) by entering the following command at the system console of either node: If your system is configured in... 7-Mode Cluster-Mode Then issue this command... cf status storage failover show 3. Take one of the following actions, depending on the result of the cf status or storage failover show command: If... Neither node is in takeover mode The partner node took over the target node The target node took over the partner node Then... Go to the next step in this procedure. The target node is in a state where you can begin removing it from the system chassis. 1. Correct the problem that caused the takeover. 2. Run the cf giveback command (if in a 7-Mode system) or storage failover giveback (if in a Cluster-Mode system) from the target node console. 3. Go back to the beginning of this procedure. 4. Take over and power down the impaired node by taking the following steps: a. Enter one of the following commands from the healthy node s console and wait for the takeover to complete: For systems operating in... 7-Mode Clustered Data ONTAP Issue the command... cf takeover In Data ONTAP 8.2 or later: storage failover takeover -ofnode impaired_node_name Chapter 4. Replacing N6000 series system devices 83

110 The impaired node is taken over and then automatically reboots and displays the Waiting for giveback... message. Note: Leave the power supplies turned on to provide power to the healthy node. b. Wait at least two minutes after takeover of the impaired node to ensure that the takeover completed successfully. c. With the impaired node showing the Waiting for giveback... message, shut it down. The method you use to shut down the node depends on whether remote management via a Remote LAN Module (RLM) is used. Is the RLM configured? Yes No Then... Log in to the impaired node's RLM and issue the following command: system power off At the impaired node's prompt, press Ctrl-C and respond Y to halt the node. Shutting down the node in a stand-alone system For a node that is not configured with a high-availability (HA) partner, you must perform a clean shutdown (ensuring that all data has been written to disk) and disconnect the power supplies. Procedure 1. Enter one of the following commands from the system console: If your system is configured as a... 7-Mode system Cluster-Mode Then issue this command... halt halt local Attention: You must perform a clean system shutdown before replacing system components to avoid losing unwritten data in the NVRAM. The NVRAM adapter has a red LED that blinks if there is unwritten data in the NVRAM. If this LED is flashing red after you enter the halt command, reboot your system and try halting it again. 2. If you are not already grounded, properly ground yourself. 3. Turn off the power supplies, unplug both power cords from the power source, and then remove the power cords. If your system uses... AC power supplies DC power supplies Then... Unplug the power cords from the power source, and then remove the power cords. Remove the power at the DC source, and remove the DC wires, if necessary. Opening the system To access components inside the controller module, you must open the system. Procedure 1. If you are not already grounded, properly ground yourself. 84 IBM System Storage: N6000 Series Hardware and Service Guide

111 2. Unplug the system cables from the controller module, as needed, and keep track of where the cables were connected. Leave the cables in the cable clips of the cable management tray so that when you reinstall the cable management tray, the cables are already aligned. 3. Grasp the cable management tray by the blue touch points on the arms, gently push one side of the tray so that the arm slides off the retaining pin, and then lift the tray off the back of the controller module and set it aside. 4. Loosen the thumbscrew on the cam handle. 5. Pull the cam handle downward and slide the controller module out of the system until it catches. 6. Push in the blue release latch on the left side of the controller module and slide the controller module out of the system. Release the latch after you can slide the controller module. Make sure that you support the bottom of the controller module with your free hand. Removing an RTC battery You must use this procedure to remove the RTC battery from the motherboard. Procedure 1. Locate the RTC battery at the right-rear of the controller module. It is the coin cell battery in the vertical holder near the CompactFlash card. Figure 14. RTC battery removal 2. With your fingernail, gently pry the retaining clip away from the top edge of the battery. Attention: Do not pry the clip too much; the clip bends if you use too much force. 3. Grasp the sides of the battery with your thumb and forefinger, lift the battery out of the holder, and set it aside. Installing an RTC battery You must use this procedure to install a Real-Time Clock (RTC) battery. Chapter 4. Replacing N6000 series system devices 85

112 Procedure 1. If you are not already grounded, properly ground yourself. 2. Remove the replacement battery from the antistatic shipping bag. 3. Locate the empty battery holder on the motherboard and insert the battery into the holder. 4. Visually inspect the battery to make sure that it is completely installed into the holder and that the polarity is correct. For battery polarity, see the illustration on the field-replaceable unit (FRU) map inside the chassis. There is also a large + on the motherboard indicating battery polarity. Reinstalling the controller module and booting the system After you replace a component within the controller module, you must reinstall the controller module in the system chassis. About this task Note: For HA pairs, the sequence in which you reinstall the controller module is especially important because it attempts to reboot as soon as you completely seat it in the chassis. Procedure 1. Close the DIMM cover and side panel, if necessary. 2. Align the end of the controller module with the opening in the chassis, if necessary, and then gently push the controller module halfway into the system. 3. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 4. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, and then push the cam handle to the closed position. If your system is in... An HA pair A stand-alone configuration Then... The node reboots as soon as you seat the controller completely into the system. 1. Press Ctrl-C to stop the boot process. 2. Enter the following command at the boot loader prompt: boot_diags 1. Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. 2. Press Ctrl-C to stop the boot process. 3. Enter the following command at the boot loader prompt: boot_diags 5. Tighten the thumbscrew beneath the cam handle. Running diagnostics on the RTC battery After installing the RTC battery you should run diagnostics. 86 IBM System Storage: N6000 Series Hardware and Service Guide

113 Procedure 1. From the diagnostics menu, select and complete the appropriate tests. See the IBM System Storage N series Diagnostics Guide for information about specific diagnostics tests you can perform. 2. Exit diagnostics by entering the following command: exit 3. Return the system to normal operation: If your system is in... An HA pair A stand-alone configuration Then issue this command... Run the cf giveback command from the partner node console. Enter the following command at the prompt to boot the system: boot_ontap Resetting the date and time on the system After you reconnect the battery and reboot Data ONTAP, you must reset the date and time on the system. Procedure 1. Display the current date on the node or nodes by entering the following command: If you have a... system operating in 7-Mode system running clustered Data ONTAP 2. Set the date by entering the following command: Then issue this command... date Note: If your system is in an HA pair, make sure that you display the date and time on the partner node and set the target node to those values. system date show Note: Make sure that you display the date and time on the other nodes in the cluster and set the target node to those values. If you have a... system operating in 7-Mode Then issue this command... date [-u] [[[CC]yy]mmddhhmm>[.<ss>]] -u sets the date and time to Greenwich Mean Time instead of the local time. CC is the first two digits of the current year. yy is the second two digits of the current year. mm is the current month. If the month is omitted, the default is the current month. dd is the current day. If the day is omitted, the default is the current day. hh is the current hour, using a 24-hour clock. mm is the current minute. ss is the current second. If the seconds are omitted, the default is 0. Chapter 4. Replacing N6000 series system devices 87

114 If you have a... system running clustered Data ONTAP Then issue this command... system date modify -node node_name -date date_and_time node_name is the target node. date_and_time is the date and time setting for the node, in the format MM/DD/YYYY HH:MM:SS. Note: The Data ONTAP System Administration Guide for 7-Mode or the Clustered Data ONTAP System Administration Guide for Cluster Administrators contains more information about setting the system date and time. The following command sets the date and time to 22 May 2009 at 9:25 a.m. on a system operating in 7-Mode: date The following command sets the date and time to 22 May 2009 at 9:25 a.m. on a system running clustered Data ONTAP: system date modify -node system1 -date "05/22/ :25:00" Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Disposing of batteries Dispose of batteries according to local regulations regarding battery recycling or disposal. For more information, see the IBM Environmental Notices and User Guide. Replacing an NVRAM battery in a system To replace an NVRAM battery in the system, you must complete a specific sequence of tasks. About this task v You can use this procedure with all versions of Data ONTAP supported by your system. v This procedure refers to HA pairs, which in releases prior to Data ONTAP 8.0 were called active/active configurations. v All other components in the system must be functioning properly; if not, contact technical support. Shutting down the node You shut down a node using different procedures, depending on whether it is a stand-alone system or part of an HA pair. Shutting down a node in an HA pair To shut down the node you must determine the status of the node and, if necessary, take over the node so that the partner continues to serve data from the node's storage. About this task Note: Leave the power supplies turned on at the end of this procedure to provide power to the partner node. 88 IBM System Storage: N6000 Series Hardware and Service Guide

115 Procedure 1. If running clustered Data ONTAP, check the status of the nodes in the cluster: a. Enter the following command at the system console of either node: cluster show The command produces output similar to the following: Node Health Eligibility node1 true true node2 true true node3 true true node4 true true 4 entries were displayed. b. Take one of the following actions, depending on the result of the command: If... All nodes show true for both health and eligibility. The impaired node shows false for health. Any nodes show false for eligibility. Any nodes other than the impaired node show false for health. Then... Proceed to Step 3. Proceed to the next step. Resolve any cluster issues as needed before continuing with this procedure. Correct the problems that cause the health issues on the nodes before continuing with this procedure. 2. Check the status of the target node (the node you want to perform maintenance on) by entering the following command at the system console of either node: If your system is configured in... 7-Mode Cluster-Mode Then issue this command... cf status storage failover show 3. Take one of the following actions, depending on the result of the cf status or storage failover show command: If... Neither node is in takeover mode The partner node took over the target node The target node took over the partner node Then... Go to the next step in this procedure. The target node is in a state where you can begin removing it from the system chassis. 1. Correct the problem that caused the takeover. 2. Run the cf giveback command (if in a 7-Mode system) or storage failover giveback (if in a Cluster-Mode system) from the target node console. 3. Go back to the beginning of this procedure. 4. Take over and power down the impaired node by taking the following steps: a. Enter one of the following commands from the healthy node s console and wait for the takeover to complete: Chapter 4. Replacing N6000 series system devices 89

116 For systems operating in... 7-Mode Clustered Data ONTAP Issue the command... cf takeover In Data ONTAP 8.2 or later: storage failover takeover -ofnode impaired_node_name The impaired node is taken over and then automatically reboots and displays the Waiting for giveback... message. Note: Leave the power supplies turned on to provide power to the healthy node. b. Wait at least two minutes after takeover of the impaired node to ensure that the takeover completed successfully. c. With the impaired node showing the Waiting for giveback... message, shut it down. The method you use to shut down the node depends on whether remote management via a Remote LAN Module (RLM) is used. Is the RLM configured? Yes No Then... Log in to the impaired node's RLM and issue the following command: system power off At the impaired node's prompt, press Ctrl-C and respond Y to halt the node. Shutting down the node in a stand-alone system For a node that is not configured with a high-availability (HA) partner, you must perform a clean shutdown (ensuring that all data has been written to disk) and disconnect the power supplies. Procedure 1. Enter one of the following commands from the system console: If your system is configured as a... 7-Mode system Cluster-Mode Then issue this command... halt halt local Attention: You must perform a clean system shutdown before replacing system components to avoid losing unwritten data in the NVRAM. The NVRAM adapter has a red LED that blinks if there is unwritten data in the NVRAM. If this LED is flashing red after you enter the halt command, reboot your system and try halting it again. 2. If you are not already grounded, properly ground yourself. 3. Turn off the power supplies, unplug both power cords from the power source, and then remove the power cords. If your system uses... AC power supplies DC power supplies Then... Unplug the power cords from the power source, and then remove the power cords. Remove the power at the DC source, and remove the DC wires, if necessary. 90 IBM System Storage: N6000 Series Hardware and Service Guide

117 Opening the system To access components inside the controller module, you must open the system. Procedure 1. If you are not already grounded, properly ground yourself. 2. Unplug the system cables from the controller module, as needed, and keep track of where the cables were connected. Leave the cables in the cable clips of the cable management tray so that when you reinstall the cable management tray, the cables are already aligned. 3. Grasp the cable management tray by the blue touch points on the arms, gently push one side of the tray so that the arm slides off the retaining pin, and then lift the tray off the back of the controller module and set it aside. 4. Loosen the thumbscrew on the cam handle. 5. Pull the cam handle downward and slide the controller module out of the system until it catches. 6. Push in the blue release latch on the left side of the controller module and slide the controller module out of the system. Release the latch after you can slide the controller module. Make sure that you support the bottom of the controller module with your free hand. Removing an NVRAM battery To remove the NVRAM battery, you must perform a specific sequence of steps. Procedure 1. Loosen the two thumbscrews on the battery cover. You might need to push down on the cover with one hand to loosen the thumbscrews. 2. Swing the cover up and away from the battery. 3. Lift the battery out of the floor of the housing, as shown in the following illustration. Chapter 4. Replacing N6000 series system devices 91

118 Figure 15. NVRAM battery location Installing an NVRAM battery To install an NVRAM battery in the controller module, you must perform a specific sequence of steps. Procedure 1. Locate the NVRAM battery, using the label on the battery cover as a reference. 2. If necessary, loosen the two thumbscrews on the battery cover. You might need to push down on the cover with one hand to loosen the thumbscrews. 3. If necessary, swing the cover up and away from the battery. 4. Align the battery socket with the edge connector in the floor of the housing. 5. Seat the battery. 6. Close the battery cover and tighten the thumbscrews. You might need to push down on the cover with one hand to tighten the thumbscrews. Reinstalling the controller module and booting the system After you replace a component within the controller module, you must reinstall the controller module in the system chassis. 92 IBM System Storage: N6000 Series Hardware and Service Guide

119 About this task Attention: For HA pairs, the sequence in which you reinstall the controller module is especially important because it attempts to reboot as soon as you completely seat it in the chassis. Procedure 1. Close the DIMM cover and side panel, if necessary. 2. Align the end of the controller module with the opening in the chassis, if necessary, and then gently push the controller module halfway into the system. 3. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 4. Push the controller module all the way into the chassis. If your system is in... An HA pair A stand-alone configuration Then... The node reboots as soon as you seat the controller module completely into the chassis. Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. 5. Firmly push the cam handle to finish seating the controller module in the system, and then push the cam handle to the closed position. 6. Tighten the thumbscrew beneath the cam handle. 7. Check the status and firmware level of the RLM by entering the following command at the system prompt: rlm status 8. Check the current RLM firmware level posted on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Refer to the list of current RLM firmware versions in the System, RLM, SP, and Disk Shelf Firmware Matrix. If your RLM firmware version is... Earlier than the latest RLM firmware version posted on the IBM N series support website The latest RLM firmware version posted on the IBM N series support website Then... Download and install the latest version of firmware from the website, following the posted installation instructions for that level of firmware. Go to the next step. Checking the status of the NVRAM battery After installing a new NVRAM battery, you should run diagnostics and check the status of the battery. Procedure 1. Reboot the system and press Ctrl-C to stop the boot process before LOADER completes loading, and then enter the following command at the boot loader prompt: boot_diags 2. Enter the following command at the main diagnostics prompt:mb 3. Select option 6, On-board NVRAM Test. Chapter 4. Replacing N6000 series system devices 93

120 4. Select option 5 from the NVRAM Environmental menu. 5. Select the third test, NVRAM Battery Test. 6. After you complete the tests, exit diagnostics. 7. Enter the following command at the prompt to boot the system: boot_ontap 8. If your system is in an HA pair, return the system to normal operation: If you have a... 7-Mode system Cluster-Mode system Then issue this command... cf giveback storage failover giveback 9. Check the status of the NVRAM battery from the system command prompt on the console by entering the following command: If you have a... 7-Mode system Cluster-Mode system Then issue this command... environ chassis NVRAM7-battery-0 print system node environment alarms show The state is shown as normal when the battery is good and fully charged. The 7-Mode output display shows the following information: Sensor name: NVRAM7-battery-0 type: voltage owner: NVRAM7-0 phys_id: 1 state: normal curr_reading: 3924mV crit.low: 3499mV warn.low: 3699mV warn.high: 4250mV crit.high: 4250mV Note: The displayed field's values can change without notice. All fields will be present and have values. 10. If the command fails or the state is not normal, reinstall the battery and run the test again. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Disposing of batteries Dispose of batteries according to local regulations regarding battery recycling or disposal. For more information, see the IBM Environmental Notices and User Guide. Replacing a DIMM in a system To replace a DIMM in the system, you must perform a specific sequence of tasks. About this task v You can use this procedure with all versions of Data ONTAP supported by your system. v This procedure refers to HA pairs, which in releases prior to Data ONTAP 8.0 were called active/active configurations. 94 IBM System Storage: N6000 Series Hardware and Service Guide

121 v All other components in the system must be functioning properly; if not, contact technical support. Shutting down the node You shut down a node using different procedures, depending on whether it is a stand-alone system or part of an HA pair. Shutting down a node in an HA pair To shut down the node you must determine the status of the node and, if necessary, take over the node so that the partner continues to serve data from the node's storage. About this task Note: Leave the power supplies turned on at the end of this procedure to provide power to the partner node. Procedure 1. If running clustered Data ONTAP, check the status of the nodes in the cluster: a. Enter the following command at the system console of either node: cluster show The command produces output similar to the following: Node Health Eligibility node1 true true node2 true true node3 true true node4 true true 4 entries were displayed. b. Take one of the following actions, depending on the result of the command: If... All nodes show true for both health and eligibility. The impaired node shows false for health. Any nodes show false for eligibility. Any nodes other than the impaired node show false for health. Then... Proceed to Step 3. Proceed to the next step. Resolve any cluster issues as needed before continuing with this procedure. Correct the problems that cause the health issues on the nodes before continuing with this procedure. 2. Check the status of the target node (the node you want to perform maintenance on) by entering the following command at the system console of either node: If your system is configured in... 7-Mode Cluster-Mode Then issue this command... cf status storage failover show 3. Take one of the following actions, depending on the result of the cf status or storage failover show command: Chapter 4. Replacing N6000 series system devices 95

122 If... Neither node is in takeover mode The partner node took over the target node The target node took over the partner node Then... Go to the next step in this procedure. The target node is in a state where you can begin removing it from the system chassis. 1. Correct the problem that caused the takeover. 2. Run the cf giveback command (if in a 7-Mode system) or storage failover giveback (if in a Cluster-Mode system) from the target node console. 3. Go back to the beginning of this procedure. 4. Take over and power down the impaired node by taking the following steps: a. Enter one of the following commands from the healthy node s console and wait for the takeover to complete: For systems operating in... 7-Mode Clustered Data ONTAP Issue the command... cf takeover In Data ONTAP 8.2 or later: storage failover takeover -ofnode impaired_node_name The impaired node is taken over and then automatically reboots and displays the Waiting for giveback... message. Note: Leave the power supplies turned on to provide power to the healthy node. b. Wait at least two minutes after takeover of the impaired node to ensure that the takeover completed successfully. c. With the impaired node showing the Waiting for giveback... message, shut it down. The method you use to shut down the node depends on whether remote management via a Remote LAN Module (RLM) is used. Is the RLM configured? Yes No Then... Log in to the impaired node's RLM and issue the following command: system power off At the impaired node's prompt, press Ctrl-C and respond Y to halt the node. Shutting down the node in a stand-alone system For a node that is not configured with a high-availability (HA) partner, you must perform a clean shutdown (ensuring that all data has been written to disk) and disconnect the power supplies. Procedure 1. Enter one of the following commands from the system console: If your system is configured as a... 7-Mode system Then issue this command... halt 96 IBM System Storage: N6000 Series Hardware and Service Guide

123 If your system is configured as a... Cluster-Mode Then issue this command... halt local Attention: You must perform a clean system shutdown before replacing system components to avoid losing unwritten data in the NVRAM. The NVRAM adapter has a red LED that blinks if there is unwritten data in the NVRAM. If this LED is flashing red after you enter the halt command, reboot your system and try halting it again. 2. If you are not already grounded, properly ground yourself. 3. Turn off the power supplies, unplug both power cords from the power source, and then remove the power cords. If your system uses... AC power supplies DC power supplies Then... Unplug the power cords from the power source, and then remove the power cords. Remove the power at the DC source, and remove the DC wires, if necessary. Opening the system To access components inside the controller module, you must open the system. Procedure 1. If you are not already grounded, properly ground yourself. 2. Unplug the system cables from the controller module, as needed, and keep track of where the cables were connected. Leave the cables in the cable clips of the cable management tray so that when you reinstall the cable management tray, the cables are already aligned. 3. Grasp the cable management tray by the blue touch points on the arms, gently push one side of the tray so that the arm slides off the retaining pin, and then lift the tray off the back of the controller module and set it aside. 4. Loosen the thumbscrew on the cam handle. 5. Pull the cam handle downward and slide the controller module out of the system until it catches. 6. Push in the blue release latch on the left side of the controller module and slide the controller module out of the system. Release the latch after you can slide the controller module. Make sure that you support the bottom of the controller module with your free hand. Removing the DIMMs To remove a DIMM, you must perform certain steps in order. Procedure 1. If you are not already grounded, properly ground yourself. 2. Open the DIMM cover. 3. Locate the DIMMs. Note: Each system memory DIMM has an LED located on the board next to each DIMM slot. The LED is illuminated if the DIMM needs to be replaced. Chapter 4. Replacing N6000 series system devices 97

124 The number of DIMMs varies, depending on your model. This illustration shows a system fully populated with DIMMs: Figure 16. Replacing DIMMs NVRAM LED NVRAM memory DIMMs NVRAM LED System memory DIMMs 4. Note the orientation of the DIMM in the socket so that you can insert the replacement DIMM in the proper orientation. 5. Push apart the latches on either side of the DIMM to release the DIMM from its slot, and then lift it out of the slot. Figure 17. Lifting the DIMM from the slot Attention: Carefully hold the DIMM by the edges to avoid pressure on the components on the DIMM circuit board. 98 IBM System Storage: N6000 Series Hardware and Service Guide

125 6. Repeat these steps to remove additional DIMMs as needed. Installing the DIMMs To install one or more DIMMs, you must perform a specific sequence of tasks. Procedure 1. If you are not already grounded, properly ground yourself. 2. Locate the slot where you are installing the new DIMM. 3. Remove the replacement DIMM from the antistatic shipping bag, hold the DIMM by the corners, and align it over the slot. The notch among the pins on the DIMM should line up with the tab in the socket. 4. Insert the DIMM straight into the slot. The DIMM fits tightly in the slot, but should go in easily. If not, realign the DIMM with the slot and reinsert it. Attention: Visually inspect the DIMM to verify that it is evenly aligned and fully inserted into the slot. The edge connector on the DIMM must make complete contact with the slot. 5. Push carefully, but firmly, on the top edge of the DIMM until the latches snap into place over the notches at the ends of the DIMM. 6. Repeat the preceding steps to install additional DIMMs as needed. 7. Close the DIMM cover. Reinstalling the controller module and booting the system After you replace a component within the controller module, you must reinstall the controller module in the system chassis. About this task Note: For HA pairs, the sequence in which you reinstall the controller module is especially important because it attempts to reboot as soon as you completely seat it in the chassis. Procedure 1. Close the DIMM cover and side panel, if necessary. 2. Align the end of the controller module with the opening in the chassis, if necessary, and then gently push the controller module halfway into the system. 3. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 4. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, and then push the cam handle to the closed position. If your system is in... An HA pair Then... The node reboots as soon as you seat the controller completely into the system. 1. Press Ctrl-C to stop the boot process. 2. Enter the following command at the boot loader prompt: boot_diags Chapter 4. Replacing N6000 series system devices 99

126 If your system is in... A stand-alone configuration Then Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. 2. Press Ctrl-C to stop the boot process. 3. Enter the following command at the boot loader prompt: boot_diags 5. Tighten the thumbscrew beneath the cam handle. Running diagnostics on the DIMM After installing a new DIMM, you should run diagnostics. Procedure 1. Enter the following command at the main diagnostics prompt to enter the memory menu:mem 2. Confirm that the total memory output is consistent with expectations. 3. Run the Comprehensive memory tests: option After you complete the tests, exit diagnostics. 5. Return the system to normal operation: If your system is in... An HA pair A stand-alone configuration Then issue this command... Run the cf giveback command (if in 7-mode) or storage failover giveback (if in cluster-mode) from the partner node console. Enter the following command at the prompt to boot the system: boot_ontap Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. Replacing PCI cards and risers in a system To replace a PCI card or riser in the system, you must perform the following tasks in order. About this task Note: If you are replacing or installing a Performance Acceleration Module (PAM) or Flash Cache module, refer to the "Installing or replacing a Flash Cache module or PAM" appendix in the IBM System Storage N series Introduction and Planning Guide. v You can use this procedure with all versions of Data ONTAP supported by your system. v This procedure refers to HA pairs, which in releases prior to Data ONTAP 8.0 were called active/active configurations. v All other components in the system must be functioning properly; if not, contact technical support. 100 IBM System Storage: N6000 Series Hardware and Service Guide

127 Shutting down the node You shut down a node using different procedures, depending on whether it is a stand-alone system or part of an HA pair. Shutting down a node in an HA pair To shut down the node you must determine the status of the node and, if necessary, take over the node so that the partner continues to serve data from the node's storage. About this task Note: Leave the power supplies turned on at the end of this procedure to provide power to the partner node. Procedure 1. If running clustered Data ONTAP, check the status of the nodes in the cluster: a. Enter the following command at the system console of either node: cluster show The command produces output similar to the following: Node Health Eligibility node1 true true node2 true true node3 true true node4 true true 4 entries were displayed. b. Take one of the following actions, depending on the result of the command: If... All nodes show true for both health and eligibility. The impaired node shows false for health. Any nodes show false for eligibility. Any nodes other than the impaired node show false for health. Then... Proceed to Step 3. Proceed to the next step. Resolve any cluster issues as needed before continuing with this procedure. Correct the problems that cause the health issues on the nodes before continuing with this procedure. 2. Check the status of the target node (the node you want to perform maintenance on) by entering the following command at the system console of either node: If your system is configured in... 7-Mode Cluster-Mode Then issue this command... cf status storage failover show 3. Take one of the following actions, depending on the result of the cf status or storage failover show command: If... Neither node is in takeover mode The partner node took over the target node Then... Go to the next step in this procedure. The target node is in a state where you can begin removing it from the system chassis. Chapter 4. Replacing N6000 series system devices 101

128 If... The target node took over the partner node Then Correct the problem that caused the takeover. 2. Run the cf giveback command (if in a 7-Mode system) or storage failover giveback (if in a Cluster-Mode system) from the target node console. 3. Go back to the beginning of this procedure. 4. Take over and power down the impaired node by taking the following steps: a. Enter one of the following commands from the healthy node s console and wait for the takeover to complete: For systems operating in... 7-Mode Clustered Data ONTAP Issue the command... cf takeover In Data ONTAP 8.2 or later: storage failover takeover -ofnode impaired_node_name The impaired node is taken over and then automatically reboots and displays the Waiting for giveback... message. Note: Leave the power supplies turned on to provide power to the healthy node. b. Wait at least two minutes after takeover of the impaired node to ensure that the takeover completed successfully. c. With the impaired node showing the Waiting for giveback... message, shut it down. The method you use to shut down the node depends on whether remote management via a Remote LAN Module (RLM) is used. Is the RLM configured? Yes No Then... Log in to the impaired node's RLM and issue the following command: system power off At the impaired node's prompt, press Ctrl-C and respond Y to halt the node. Shutting down the node in a stand-alone system For a node that is not configured with a high-availability (HA) partner, you must perform a clean shutdown (ensuring that all data has been written to disk) and disconnect the power supplies. Procedure 1. Enter one of the following commands from the system console: If your system is configured as a... 7-Mode system Cluster-Mode Then issue this command... halt halt local 102 IBM System Storage: N6000 Series Hardware and Service Guide

129 Attention: You must perform a clean system shutdown before replacing system components to avoid losing unwritten data in the NVRAM. The NVRAM adapter has a red LED that blinks if there is unwritten data in the NVRAM. If this LED is flashing red after you enter the halt command, reboot your system and try halting it again. 2. If you are not already grounded, properly ground yourself. 3. Turn off the power supplies, unplug both power cords from the power source, and then remove the power cords. If your system uses... AC power supplies DC power supplies Then... Unplug the power cords from the power source, and then remove the power cords. Remove the power at the DC source, and remove the DC wires, if necessary. Opening the system To access components inside the controller module, you must open the system. Procedure 1. If you are not already grounded, properly ground yourself. 2. Unplug the system cables from the controller module, as needed, and keep track of where the cables were connected. Leave the cables in the cable clips of the cable management tray so that when you reinstall the cable management tray, the cables are already aligned. 3. Grasp the cable management tray by the blue touch points on the arms, gently push one side of the tray so that the arm slides off the retaining pin, and then lift the tray off the back of the controller module and set it aside. 4. Loosen the thumbscrew on the cam handle. 5. Pull the cam handle downward and slide the controller module out of the system until it catches. 6. Push in the blue release latch on the left side of the controller module and slide the controller module out of the system. Release the latch after you can slide the controller module. Make sure that you support the bottom of the controller module with your free hand. Removing a PCIe card You must use this procedure to remove a PCIe card from the system. Procedure 1. Loosen the thumbscrew on the controller module side panel. 2. Swing the side panel open until it comes off the controller module. Chapter 4. Replacing N6000 series system devices 103

130 2 1 Figure 18. Replacing a PCIe card and riser 1 2 PCIe card Riser 3. Remove the PCIe card from the controller module and set it aside. 4. Repeat the above steps as needed. Installing a PCIe card You must use this procedure to install a PCI card in the system. Procedure 1. Open the controller module side panel, if necessary, slide off the PCI card filler plate, and install the PCI card. Be sure that you properly align the card in the slot and exert even pressure on the card when seating it in the socket. The adapter must be fully and evenly seated in the slot. 2. Repeat as needed for additional cards. 3. Close the side panel and tighten the thumbscrew. Removing a riser You must use this procedure to remove the riser from the controller. Before you begin You must have removed any PCIe cards from the riser. Procedure Using the PCIe Card label on the controller DIMM cover for reference, loosen the riser thumbscrew, and then pull the riser up and out of the sockets. 104 IBM System Storage: N6000 Series Hardware and Service Guide

131 Installing a riser You must use this procedure to install the riser in the controller. Procedure 1. While grounded, align the riser with the guide slots on the chassis, and then firmly push down until the riser is in place. 2. Examine the riser to make sure that it is seated squarely and completely in the sockets. If it is not seated squarely, repeat the previous step to reinstall it. 3. Tighten the riser thumbscrew. 4. Open the controller side panel, if necessary, to reinstall the PCIe cards, and then close the side panel and tighten the thumbscrew. Reinstalling the controller module and booting the system After you replace a component within the controller module, you must reinstall the controller module in the system chassis. About this task Note: For HA pairs, the sequence in which you reinstall the controller module is especially important because it attempts to reboot as soon as you completely seat it in the chassis. Procedure 1. Close the DIMM cover and side panel, if necessary. 2. Align the end of the controller module with the opening in the chassis, if necessary, and then gently push the controller module halfway into the system. 3. Reinstall the cable management tray and recable the system, as needed. When recabling, remember to reinstall the media converters (SFPs) if you are using fiber cables. 4. Push the controller module all the way into the system, firmly push the cam handle to finish seating it, and then push the cam handle to the closed position. If your system is in... An HA pair A stand-alone configuration Then... The node reboots as soon as you seat the controller completely into the system. 1. Press Ctrl-C to stop the boot process. 2. Enter the following command at the boot loader prompt: boot_diags 1. Reconnect the power cables to the power supplies and to the power sources, and then turn on the power. 2. Press Ctrl-C to stop the boot process. 3. Enter the following command at the boot loader prompt: boot_diags 5. Tighten the thumbscrew beneath the cam handle. Running diagnostics on a PCIe card After installing a PCIe card, you should run diagnostics. Chapter 4. Replacing N6000 series system devices 105

132 Procedure 1. From the diagnostics menu, select and complete the appropriate tests. See the IBM System Storage N series Diagnostics Guide for information about specific diagnostics tests you can perform. 2. After you complete the tests, exit diagnostics. 3. Exit the boot loader. 4. Return the system to normal operation: If your system is in a... HA pair Stand-alone configuration Then issue this command Run the cf giveback command (if in a 7-Mode system) or storage failover giveback (if in a Cluster-Mode system) from the partner node console. 2. On the console of the newly installed controller module, select option 1 from the displayed menu to boot Data ONTAP: Normal Boot (option 1) Enter the following command at the prompt to boot the system: boot_ontap Configuring newly added Ethernet ports for cluster or management use (Data ONTAP 8.2 Cluster-Mode) When you install an Ethernet card in a Data ONTAP for Cluster-Mode, the ports are automatically configured as data ports. To use the ports as cluster ports or management ports, you must manually reconfigure them. Procedure Enter the following command to change the port configuration: network port modify -node node -port port-name -role port-role port-role can be mgmt, cluster, or data. Note: For more information, see the Data ONTAP 8.2 Cluster-Mode Network and File Access Management Guide. This document, and other Data ONTAP documentation, is available on the IBM N series support website, which is accessed and navigated as described in Websites on page xx. Completing the replacement process Return the failed part to IBM. Contact IBM Service and Support at IBM-SERV ( ) for more information on the return procedure. 106 IBM System Storage: N6000 Series Hardware and Service Guide

133 Appendix A. Recommended power line sizes This appendix discusses how to determine the power line lengths running from your N series system to the power source. Recommended AC power line sizes Longer AC power feeds need to be properly designed to preserve voltage levels to the equipment. The wiring from the breaker panel to the power strip, which supplies power to your N6000 series system and storage expansion units, can often exceed 50 feet. Note: Total AC wire length = breaker to wall or ceiling outlet + extension cable or ceiling drop. The following table lists the recommended conductor size for 2% voltage drop for a particular distance in feet (taken from the Radio Engineer's Handbook). Table V, single phase recommended conductor sizes 110V, single-phase 20A circuit 30A circuit 40A circuit 50A circuit 25 feet 12 AWG 10 AWG 8 AWG 8 AWG 50 feet 8 AWG 6 AWG 6 AWG 4 AWG 75 feet 6 AWG 4 AWG 4 AWG 2 AWG Table V, single phase recommended conductor sizes 220V, single-phase 20A circuit 30A circuit 40A circuit 50A circuit 25 feet 14 AWG 12 AWG 12 AWG 10 AWG 50 feet 12 AWG 10 AWG 8 AWG 8 AWG 75 feet 10 AWG 8 AWG 6 AWG 6 AWG The following table list the approximate equivalent wire gauge (American Wire Gauge (AWG) to Harmonized Cordage). Table 21. American Wire Gage to Harmonized Cordage equivalents AWG Harmonized, mm-mm mm-mm = millimeter squared Copyright IBM Corp. 2008,

134 108 IBM System Storage: N6000 Series Hardware and Service Guide

135 Appendix B. FRU/CRU and power cord list for N series products This appendix contains information about FRU/CRUs and power cords for N series products. FRU/CRU list for N series products For the most current FRU/CRU list for your N series product, access the IBM N series support website, as described in Websites on page xx, and refer to the FRU (Field Replaceable Units) lists. Power cord list for N series products The following list details the power cord feature codes (FCs) for N series products. FC 9000 (All countries) Power cord, Rack PDU v 27 inches v Rated 250 V/15 A v Product end uses C14; PDU end uses C13. FC 9001 Europe and others Provides power cords for Austria, Belgium, Bolivia, Bulgaria, Chile, Croatia, Czech Republic, Egypt, Estonia, European Union, Finland, France, Germany, Greece, Hungary, Iceland, Indonesia, Latvia, Lebanon, Lithuania, Luxemburg, Morocco, Netherlands, Norway, Peru, Poland, Portugal, Romania, Russia, Slovakia, Slovenia, Spain, Suriname, Sweden, Turkey v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 211 (CEE 7-VII) designed for V ac input. FC 9002 United Kingdom and others Provides power cords for United Kingdom, Costa Rica, Cyprus, Guyana, Hong Kong, Ireland, Kuwait, Malta, Oman, Singapore, Sri Lanka v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 210 (13A fuse) designed for V ac input. FC 9003 Japan Provides power cords for Japan v 1.83 m (6 feet), unshielded, rated 125 V/15 A. v Attached plug EL 302 (JIS C3306) designed for V ac input. FC 9004 U.S., Six Feet (2 m) Provides power cords for U.S., Canada, Mexico, Belize, Columbia, Ecuador, El Salvador, Guatemala, Honduras, Korea, Nicaragua, Panama, Philippines, Puerto Rico, Saudi Arabia, Thailand, Venezuela v 1.83 m (6 feet), unshielded, rated 125 V/15 A. v Attached plug EL 302 (Nema 5-15P) designed for V ac input. FC 9005 Australia, New Zealand Provides power cords for Australia, New Zealand, Uruguay v 2.5 m (9 feet), unshielded, rated 250 V/10 A. Copyright IBM Corp. 2008,

136 v Attached plug EL 206 (AS 3112) designed for V ac input. FC 9006 Switzerland, Liechtenstein Provides power cords for Switzerland, Liechtenstein v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 203 (SEV 1011) designed for V ac input. FC 9007 Argentina Provides power cords for Argentina v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 219 (IRAM 2073) designed for V ac input. FC 9008 China Provides power cords for China v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 602 (GB 2099/GB 1002) designed for V ac input. FC 9009 Denmark Provides power cords for Denmark v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 213 (DHCR D1) designed for V ac input. FC 9010 India, Pakistan, South Africa Provides power cords for India, Macau, Pakistan, South Africa v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 208 (BS 164-1, BS 546) designed for V ac input. FC 9011 Israel Provides power cords for Israel v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 212 (SI 32) designed for V ac input. FC 9012 Italy Provides power cords for Italy v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 502 (CEI 23-16) designed for V ac input. FC 9013 North America (250 V) Provides power cords for U.S. v 1.83 m (6 feet), unshielded, rated 250 V/15 A. v Attached plug EL 309 (NEMA 6-15P) designed for V ac input. FC 9014 Brazil Provides power cords for Brazil v 2.5 m (9 feet), unshielded, rated 250 V/10 A. v Attached plug EL 211 (NBR 6147/2000) designed for V ac input FC 9015 Taiwan Provides 125 V power cords for Taiwan v 2.5 m (9 feet), unshielded, rated 125 V/15 A. v Attached plug EL 302 (CNS ) designed for V ac input. FC 9016 Taiwan (250 V) Provides 250 V power cords for Taiwan v 1.83 m (6 feet), unshielded, rated 250 V/10 A. 110 IBM System Storage: N6000 Series Hardware and Service Guide

137 v Attached plug EL 610 (CNS 10917, CNS 690) designed for 250 V ac input. Appendix B. FRU/CRU and power cord list for N series products 111

138 112 IBM System Storage: N6000 Series Hardware and Service Guide

139 Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe on any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-ibm product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, N.Y U.S.A. For additional information, visit the web at: contact/ The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level Copyright IBM Corp. 2008,

140 systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurement may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. If you are viewing this information in softcopy, the photographs and color illustrations may not appear. Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. A complete and current list of other IBM trademarks is available on the web at NetApp, the Network Appliance logo, the bolt design, NetApp the Network Appliance Company, Data ONTAP, DataFabric, FAServer, FilerView, MultiStore, NearStore, NetCache, SecureShare, SnapManager, SnapMirror, SnapMover, SnapRestore, SnapVault, SyncMirror, and WAFL are registered trademarks of Network Appliance, Inc. in the United States, and/or other countries. gfiler, Network Appliance, SnapCopy, Snapshot, and The Evolution of Storage are trademarks of Network Appliance, Inc. in the United States and/or other countries and registered trademarks in some other countries. ApplianceWatch, BareMetal, Camera-to-Viewer, ComplianceClock, ComplianceJournal, ContentDirector, ContentFabric, EdgeFiler, FlexClone, FlexVol, FPolicy, HyperSAN, InfoFabric, LockVault, Manage ONTAP, NOW, NetApp on the web, ONTAPI, RAID-DP, RoboCache, RoboFiler, SecureAdmin, Serving Data by Design, SharedStorage, Simulate ONTAP, Smart SAN, SnapCache, SnapDirector, SnapDrive, SnapFilter, SnapLock, SnapMigrator, SnapSuite, SnapValidator, SohoFiler, vfiler, VFM, Virtual File Manager, VPolicy, and Web Filer are trademarks of Network Appliance, Inc. in the United States and other countries. NetApp Availability Assurance and NetApp ProTech Expert are service marks of Network Appliance, Inc. in the United States. Spinnaker Networks, the Spinnaker Networks logo, SpinAccess, SpinCluster, SpinFS, SpinHA, SpinMove, and SpinServer are registered trademarks of Spinnaker Networks, LLC in the United States and/or other countries. SpinAV, SpinManager, SpinMirror, SpinRestore, SpinShot, and SpinStor are trademarks of Spinnaker Networks, LLC in the United States and/or other countries. Network Appliance is a licensee of the CompactFlash and CF Logo trademarks. Network Appliance NetCache is certified RealSystem compatible. Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. 114 IBM System Storage: N6000 Series Hardware and Service Guide

141 Important notes Electronic emission notices Other company, product, or service names may be trademarks or service marks of others. Processor speeds indicate the internal clock speed of the microprocessor; other factors also affect application performance. CD-ROM drive speeds list the variable read rate. Actual speeds vary and are often less than the maximum possible. When referring to processor storage, real and virtual storage, or channel volume, KB stands for approximately 1000 bytes, MB stands for approximately bytes, and GB stands for approximately bytes. When referring to hard disk drive capacity or communications volume, MB stands for bytes, and GB stands for bytes. Total user-accessible capacity may vary depending on operating environments. Maximum internal hard disk drive capacities assume the replacement of any standard hard disk drives and population of all hard disk drive bays with the largest currently supported drives available from IBM. Maximum memory may require replacement of the standard memory with an optional memory module. IBM makes no representation or warranties regarding non-ibm products and services that are ServerProven, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. These products are offered and warranted solely by third parties. IBM makes no representations or warranties with respect to non-ibm products. Support (if any) for the non-ibm products is provided by the third party, not IBM. Some software may differ from its retail version (if available), and may not include user manuals or all program functionality. Attention: In compliance with the GNU General Public License (GPL), Version 2, June 1991, a complete machine-readable copy of the source code for the relevant source code portions of the Remote LAN Module (RLM) Firmware that are covered by the GPL, is available from ftp://ftp.netapp.com/frm-ntap/opensource/. The following statements apply to this product. The statements for other products intended for use with this product will appear in their accompanying manuals. Federal Communications Commission (FCC) Class A Statement This equipment has been tested and complies with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Notices 115

142 Operation of this equipment in a residential area is likely to cause harmful interference, in which case the user will be required to correct the interference at his own expense. Properly shielded and grounded cables and connectors must be used in order to meet FCC emission limits. IBM is not responsible for any radio or television interference caused by using other than recommended cables and connectors or by unauthorized changes or modifications to this equipment. Unauthorized changes or modifications could void the user's authority to operate the equipment. This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) this device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation. Industry Canada Class A Emission Compliance Statement This Class A digital apparatus complies with Canadian ICES-003. Avis de conformité à la réglementation d'industrie Canada Cet appareil numérique de la classe A est conform à la norme NMB-003 du Canada. European Union EMC Directive Conformance Statement This product is in conformity with the protection requirements of EU Council Directive 2004/108/EC on the approximation of the laws of the Member States relating to electromagnetic compatibility. IBM cannot accept responsibility for any failure to satisfy the protection requirements resulting from a non-recommended modification of the product, including the fitting of non-ibm option cards. Attention: This is an EN55022 Class A product. In a domestic environment this product may cause radio interference in which case the user may be required to take adequate measures. Responsible manufacturer: International Business Machines Corp. New Orchard Road Armonk, New York Tel: European community contact: IBM Deutschland GmbHTechnical Regulations, Department M456 IBM-Allee 1, Ehningen, Germany Germany Electromagnetic Compatibility Directive Deutschsprachiger EU Hinweis: Hinweis für Geräte der Klasse A EU-Richtlinie zur Elektromagnetischen Verträglichkeit Dieses Produkt entspricht den Schutzanforderungen der EU-Richtlinie 2004/108/EG zur Angleichung der Rechtsvorschriften über die elektromagnetische Verträglichkeit in den EU-Mitgliedsstaaten und hält die Grenzwerte der EN Klasse A ein. Um dieses sicherzustellen, sind die Geräte wie in den Handbüchern beschrieben zu installieren und zu betreiben. Des Weiteren dürfen auch nur von der IBM empfohlene Kabel angeschlossen werden. IBM übernimmt keine Verantwortung für die Einhaltung der Schutzanforderungen, wenn das Produkt ohne Zustimmung der 116 IBM System Storage: N6000 Series Hardware and Service Guide

143 IBM verändert bzw. wenn Erweiterungskomponenten von Fremdherstellern ohne Empfehlung der IBM gesteckt/eingebaut werden. EN Klasse A Geräte müssen mit folgendem Warnhinweis versehen werden: "Warnung: Dieses ist eine Einrichtung der Klasse A. Diese Einrichtung kann im Wohnbereich Funk-Störungen verursachen; in diesem Fall kann vom Betreiber verlangt werden, angemessene Maßnahmen zu ergreifen und dafür aufzukommen." Deutschland: Einhaltung des Gesetzes über die elektromagnetische Verträglichkeit von Geräten Dieses Produkt entspricht dem "Gesetz über die elektromagnetische Verträglichkeit von Geräten (EMVG)". Dies ist die Umsetzung der EU-Richtlinie 2004/108/EG in der Bundesrepublik Deutschland. Zulassungsbescheinigung laut dem Deutschen Gesetz über die elektromagnetische Verträglichkeit von Geräten (EMVG) (bzw. der EMC EG Richtlinie 2004/108/EG) für Geräte der Klasse A Dieses Gerät ist berechtigt, in Übereinstimmung mit dem Deutschen EMVG das EG-Konformitätszeichen - CE - zu führen. Verantwortlich für die Einhaltung der EMV Vorschriften ist der Hersteller: International Business Machines Corp. New Orchard Road Armonk, New York Tel: Der verantwortliche Ansprechpartner des Herstellers in der EU ist: IBM Deutschland GmbHTechnical Regulations, Abteilung M456 IBM-Allee 1, Ehningen, Germany Generelle Informationen: Das Gerät erfüllt die Schutzanforderungen nach EN und EN Klasse A. People's Republic of China Class A Electronic Emission Statement Japan Voluntary Control Council for Interference (VCCI) Class A Statement Notices 117

144 Translation: This is a Class A product based on the standard of the VCCI Council. If this equipment is used in a domestic environment, radio interference may occur, in which case, the user may be required to take corrective actions. Japan Electronics and Information Technology Industries Association (JEITA) Statement Japanese Electronics and Information Technology Industries Association (JEITA) Confirmed Harmonics Guideline (products less than or equal to 20 A per phase). Korea Communications Commission (KCC) Class A Statement Please note that this equipment has obtained EMC registration for commercial use. In the event that it has been mistakenly sold or purchased, please exchange it for equipment certified for home use. Russia Electromagnetic Interference (EMI) Class A Statement rusemi jjieta1 Taiwan Class A Electronic Emission Statement 118 IBM System Storage: N6000 Series Hardware and Service Guide