Sun Blade 6000 Modular System

Transcription

1 Sun Blade 6000 Modular System Service Manual Part No.: December 2010, Revision A

2 Copyright 2007, 2010, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related software documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT RIGHTS. Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR , Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications which may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open Company, Ltd. This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services. Copyright 2007, 2010, Oracle et/ou ses affiliés. Tous droits réservés. Ce logiciel et la documentation qui l accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d utilisation et de divulgation. Sauf disposition de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, breveter, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé que ce soit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d interopérabilité avec des logiciels tiers ou tel que prescrit par la loi. Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu elles soient exemptes d erreurs et vous invite, le cas échéant, à lui en faire part par écrit. Si ce logiciel, ou la documentation qui l accompagne, est concédé sous licence au Gouvernement des Etats-Unis, ou à toute entité qui délivre la licence de ce logiciel ou l utilise pour le compte du Gouvernement des Etats-Unis, la notice suivante s applique : U.S. GOVERNMENT RIGHTS. Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR , Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA Ce logiciel ou matériel a été développé pour un usage général dans le cadre d applications de gestion des informations. Ce logiciel ou matériel n est pas conçu ni n est destiné à être utilisé dans des applications à risque, notamment dans des applications pouvant causer des dommages corporels. Si vous utilisez ce logiciel ou matériel dans le cadre d applications dangereuses, il est de votre responsabilité de prendre toutes les mesures de secours, de sauvegarde, de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité. Oracle Corporation et ses affiliés déclinent toute responsabilité quant aux dommages causés par l utilisation de ce logiciel ou matériel pour ce type d applications. Oracle et Java sont des marques déposées d Oracle Corporation et/ou de ses affiliés.tout autre nom mentionné peut correspondre à des marques appartenant à d autres propriétaires qu Oracle. AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques déposées d Advanced Micro Devices. Intel et Intel Xeon sont des marques ou des marques déposées d Intel Corporation. Toutes les marques SPARC sont utilisées sous licence et sont des marques ou des marques déposées de SPARC International, Inc. UNIX est une marque déposée concédée sous licence par X/Open Company, Ltd. Ce logiciel ou matériel et la documentation qui l accompagne peuvent fournir des informations ou des liens donnant accès à des contenus, des produits et des services émanant de tiers. Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l accès à des contenus, produits ou services tiers, ou à leur utilisation Please Recycle

3 Contents Preface vii 1. Fault Management Diagnosing Chassis Faults 1 1 How to Diagnose System Faults Fault LEDs Chassis Fault LEDs Power Supply Fault LEDs Rear Fan Fault LEDs Replacing a Faulty Component 1 5 How To Replace a Faulty Component Preparing for Service Identifying Chassis Models and Replacement Components Identifying the Chassis Model Identifying the Chassis Using ILOM Identifying the Chassis Using CMM Visual Inspection 2 3 How to Identify the Chassis Using CMM Visual Inspection Identifying Chassis Replacement Components Component Types and Service Classifications 2 6 iii

4 2.3 Locating System Components Powering Off the System Before Service 2 10 How to Power Off the System Using the Web Interface 2-11 How to Power Off the System Using the CLI Required Installation Tools and ESD 2 12 How to Attach the Antistatic Wrist Strap Replacing System Components Customer-Replaceable Units Replacing Power Supplies 3 2 How to Remove a Power Supply Module 3-2 How to Install a Power Supply Module Replacing the Front Fan Modules 3 5 How to Remove a Front Fan Module 3-5 How to Install a Front Fan Module Replacing Server or Storage Modules 3 8 How to Remove a Server or Storage Module 3-8 How to Install a Server or Storage Module Replacing the Front Indicator Module 3 11 How to Remove the Front Indicator Module 3-11 How to Install the Front Indicator Module Replacing NEMs 3 13 How to Remove a NEM 3-14 How to Install a NEM Replacing PCIe EMs 3 16 How to Remove a PCIe EM 3-17 How to Install a PCIe EM Replacing the Rear Fan Modules 3 20 How to Remove a Rear Fan Module 3-20 iv Sun Blade 6000 Modular System Service Manual December 2010

5 How to Install a Rear Fan Module Replacing the Fan Board 3 22 How to Remove a Fan Board 3-23 How to Install a Fan Board Replacing the CMM 3 25 How to Remove the CMM 3-25 How to Install the CMM Field-Replaceable Units Replacing the Midplane 3 28 How to Remove the Midplane 3-29 How to Install the Midplane 3-31 How to Update FRU Information After Replacing the Midplane Replacing the Internal Power Cables 3 35 How to Remove an Internal Power Cable 3-35 How to Install an Internal Power Cable Servicetool FRU Update Procedure Powering on the System After Service Power Cord Connections Powering On the System 3 40 How to Power On the System Automatically 340 How to Power on the System Using the Web Interface 341 How to Power on the System Using the CLI Interface 341 A. LED Indicators and Buttons A 1 Index Index 1 Contents v

6 vi Sun Blade 6000 Modular System Service Manual December 2010

7 Preface This Sun Blade 6000 Modular System Service Manual contains information and procedures for maintaining and upgrading the servers. Product Updates You can navigate to and download product updates for the Sun Blade 6000 modular system at: Related Documentation Documentation for the Sun Blade 6000 modular system is on the web at: The documentation for the Sun Blade 6000 modular system is divided by function into the following collections: System Documentation This collection contains the basic documentation for the system, including installation, service, safety and compliance, and product notes. vii

8 ILOM Documentation This collection contains documentation for the chassis service processor, called the Chassis Monitoring Module Integrated Lights-Out Manager (CMM ILOM). It also provides documentation for the Integrated Lights- Out Manager (ILOM), which is the service processor used by most (but not all) server modules. Network Express Module Documentation These collections provide documentation for their respective NEMs. Translated versions of some of these documents are available on the web site described above in French, Simplified Chinese, Traditional Chinese, Korean, and Japanese. English documentation is revised more frequently and might be more upto-date than the translated documentation. Documentation, Support, and Training Function Documentation Support Training URL Documentation Feedback Submit comments about this document by clicking the Feedback[+] link at Include the title and part number of your document with your feedback: Sun Blade 6000 Modular System Product Notes, part number, viii Sun Blade 6000 Modular System Service Manual December 2010

9 CHAPTER 1 Fault Management This chapter contains information about the following topics related to fault management on the Sun Blade 6000 modular system: Section 1.1, Diagnosing Chassis Faults on page 1-1 Section 1.2, Fault LEDs on page 1-2 Section 1.3, Replacing a Faulty Component on page Diagnosing Chassis Faults The following types of faults can be diagnosed using system LEDs and ILOM: External chassis faults Rear fan faults Power supply and power supply fan faults Note If a Service Actioin Required LED is illuminated on a server or storage module (blade), network express module (NEM), or PCIe ExpressModule (PCIe EM) or you identify a blade or NEM fault in ILOM, refer to the component documentation for more information about diagnosing the fault. How to Diagnose System Faults 1. Check system service LEDs. When the amber Service Required LED is lit for a particular component, a fault is indicated for this component. See LEDs for the locations of the faults. See Fault LEDs on page

10 2. To obtain information about the fault, log into the CMM web or CLI. You can view fault information in the following ways: View component sensor readings View fault status in ILOM fault management View the ILOM system event log For information about how to view fault status in ILOM, refer to the Oracle Integrated Lights Out Manager (ILOM) Supplement for the Sun Blade 6000 and 6048 Server Modules and to to the ILOM documentation at Fault LEDs When a component experiences a hardware failure (enters a fault state), the Service Action Required (amber) LED on that component is illuminated. In addition, the Service Action Required LEDs on the system chassis (both front and back) are illuminated when any system component is in a fault state. The following sections contain more information about system service LEDs: Chassis Fault LEDs on page 1-2 Power Supply Fault LEDs on page 1-4 Rear Fan Fault LEDs on page 1-5 Note This section only contains information about the service LEDs listed above. Additional information on system LEDs is in LED Indicators and Buttons on page A Chassis Fault LEDs There are two types of chassis fault LEDs: The chassis Service Action Required LED illuminates when there is a fault on a chassis component. The chassis Temperature Fail LED lights when an external condition exists that can adversely affect the operation of the system. Check the CMM ILOM for ambient temperature fault information. FIGURE 1-1 and FIGURE 1-2 shows the location of the LEDs on the front and rear of the chassis. 1-2 Sun Blade 6000 Modular System Service Manual December 2010

11 FIGURE 1-1 Front Chassis LEDs Locate LED Service Action Required LED Power/OK LED Temperature Fail LED FIGURE 1-2 Rear Chassis LEDs Locate LED Service Action Required LED Power/OK LED Temperature Fail LED CMM power/ok LED Chapter 1 Fault Management 1-3

12 Power Supply Fault LEDs FIGURE 1-3 shows the power supply LED locations. FIGURE 1-3 Power Supply LED Locations PSU Service Action Required LED DC LED AC-1 LED AC-0 LED Front Fan Service Action Required LED TABLE 1-1 shows the operation of the LEDs during normal operation or when a fault has occurred. TABLE 1-1 Power Supply Fault LED Functions Condition AC-0 LED (Green) AC-1 LED (Green) DC LED (Green) PSU Service LED (Yellow) Fan Service LED (Yellow) Normal operation ON ON ON OFF OFF Over current, over voltage, or over temperature warning fault ON ON OFF ON OFF AC 0 failed OFF ON OFF OFF OFF AC 1 failed ON OFF OFF OFF OFF Front fan failed ON ON ON OFF ON 1-4 Sun Blade 6000 Modular System Service Manual December 2010

13 Rear Fan Fault LEDs The rear fan fault LEDs indicate when a failure has occurred on a fan module. The source of the failure could be mechanical, electrical, or the result of a rear fan board controller failure. FIGURE 1-4 shows the rear fan LED locations. FIGURE 1-4 Rear Fan LED Locations Fan Service Action Required LED 1.3 Replacing a Faulty Component When a hardware failure occurs, the appropriate and recommended method for clearing the fault is to replace the failed component. How To Replace a Faulty Component 1. Determine which system component has experienced a hardware failure. Look at the Service Action Required LEDs and ILOM to get information about the component failure. See Section 1.1, Diagnosing Chassis Faults on page 1-1. Chapter 1 Fault Management 1-5

14 2. Remove and replace the failed component. Refer to the instructions in Chapter To confirm that the fault has been cleared, check the component LEDs. 1-6 Sun Blade 6000 Modular System Service Manual December 2010

15 CHAPTER 2 Preparing for Service This chapter contains information and procedures for servicing the Sun Blade 6000 modular system hardware, including component removal and replacement procedures. The following sections are in this chapter: Section 2.1, Identifying Chassis Models and Replacement Components on page 2-1 Section 2.2, Component Types and Service Classifications on page 2-6 Section 2.3, Locating System Components on page 2-7 Section 2.4, Powering Off the System Before Service on page 2-10 Section 2.5, Required Installation Tools and ESD on page Identifying Chassis Models and Replacement Components There are now two chassis models available for the Sun Blade 6000 modular system, the A90-B and A90-D. Make sure that you know which chassis you have. Some replaceable components are not interchangeable between the two chassis. Review the following sections for information about identifying the chassis and replacement components: Identifying the Chassis Model on page 2-2 Identifying Chassis Replacement Components on page

16 2.1.1 Identifying the Chassis Model Use one of the following methods to identify the chassis model: Identifying the Chassis Using ILOM on page 2-2 Identifying the Chassis Using CMM Visual Inspection on page Identifying the Chassis Using ILOM You can use the CMM number to identify the chassis type by using one of the following procedures: How to Find the CMM Part Number Using the ILOM Web Interface on page 2-2 How to Find the CMM Part Number Using the ILOM CLI on page 2-2 How to Find the CMM Part Number Using the ILOM Web Interface 1. Log in to the ILOM web interface. The Chassis View page appears. 2. Click on CMM in the left navigation panel. 3. Click on the System Information -> Components tab. 4. Click on /CH/CMM in the Component Name column. A dialog displays with the CMM FRU part information. If the FRU Part Number is xx, you have the A90-B chassis If the FRU Part Number is xx, you have the A90-D chassis. How to Find the CMM Part Number Using the ILOM CLI 1. Log into the ILOM CLI interface. 2. Type the following command to show CMM properties: -> show /CH/CMM Output similar to the following appears: 2-2 Sun Blade 6000 Modular System Service Manual December 2010

17 /CH/CMM Targets: SP NET0 T_AMB Properties: type = Chassis Monitoring Module ipmi_name = CMM fru_name = CMM fru_part_number = fru_serial_number = 1005LCB-0713YD013K Commands: cd reset show In the properties section, view the fru_part_number = information field. If fru_part_number = xx, you have the A90-B chassis If fru_part_number = xx, you have the A90-D chassis Identifying the Chassis Using CMM Visual Inspection The A90-B chassis CMM ( ) and the A90-D CMM ( ) have LEDs and connectors in different positions. Use the following procedure to visually identify the CMM. How to Identify the Chassis Using CMM Visual Inspection 1. Locate the CMM in the chassis. Chapter 2 Preparing for Service 2-3

18 FIGURE 2-1 Rear View of Sun Blade Modular System Chassis PCIe ExpressModules (PCIe EMs) Network express modules (NEMs) Chassis monitoring module (CMM) Power connectors Rear fan modules 2. Compare the CMM in your chassis to the illustrations shown in FIGURE Sun Blade 6000 Modular System Service Manual December 2010

19 FIGURE 2-2 CMM LEDs and ports NET MGT port 0 NET MGT port 1 SER MGT port A90-D Chassis CMM ( ) SER MGT port NET MGT port 1 NET MGT port 0 A90-B Chassis CMM ( ) Chapter 2 Preparing for Service 2-5

20 2.1.2 Identifying Chassis Replacement Components The following table lists the components that are unique to each chassis. When replacing these components, make sure to replace the component that goes with the correct chassis. TABLE 2-1 Component to Chassis Correspondence Component A90-B A90-D Power supply Rear fan CMM Midplane assembly Component Types and Service Classifications All components in the Sun Blade modular system are removable and serviceable by a single person. There are two types of serviceable components: CRU - Customer replaceable units can be removed and replaced by any qualified service provider. FRU - Field-replaceable units must be removed and installed only by authorized Oracle service personnel. Components also have three service classifications: Hot-Plug - Using the proper software commands, you can install or remove hot-plug components while the system is running. Hot-Swap - Hot-swappable components can be installed or removed while the system is running, without using any software commands. Cold-Swap - Power must be removed from the system before installing or removing the components. 2-6 Sun Blade 6000 Modular System Service Manual December 2010

21 Note See the individual component documentation for hot plug insert and removal details. The individual component documents supersede this document in hot pluggable specifics. TABLE 2-2 Description of Components CRU Component Type Service Classification Server modules (blades) CRU Hot plug Power supply modules CRU Hot swap Front fan modules CRU Hot swap Front indicator module (FIM) CRU Hot swap Chassis management module (CMM) CRU Hot swap Network express modules (NEMs) CRU Hot swap. hot plug, or cold swap, depending on NEM type PCIe ExpressModules (PCIe EMs) CRU Hot plug Rear fan modules CRU Hot swap Rear fan boards CRU Cold swap Midplane assembly FRU Cold swap Internal power cables FRU Cold swap 2.3 Locating System Components The following illustrations show the front, rear, and internal components for the Sun Blade 6000 chassis. Chapter 2 Preparing for Service 2-7

22 FIGURE 2-3 Front View of Sun Blade Modular System Chassis Power supplies Front indicator module (FIM) Server or storage modules (blades) 2-8 Sun Blade 6000 Modular System Service Manual December 2010

23 FIGURE 2-4 Rear View of Sun Blade Modular System Chassis PCIe ExpressModules (PCIe EMs) Network express modules (NEMs) Chassis monitoring module (CMM) Power connectors Rear fan modules FIGURE 2-5 Internal Components Internal power cables System midplane Front fan modules Chapter 2 Preparing for Service 2-9

24 2.4 Powering Off the System Before Service Although most components of the Sun Blade 6000 modular system can be removed while the system is powered on, you must remove power from the system to service the following components: Midplane Internal power cables Rear fan controller board Use one of the following procedures to power off the system using ILOM: How to Power Off the System Using the Web Interface on page 2-11 How to Power Off the System Using the CLI on page 2-11 How to Power Off the System Using the Web Interface 1. Log into the ILOM Web interface as administrator. The Chassis View page appears. 2. Click on CMM in the left navigation panel. 3. Navigate to Remote Control -> Remote Power Control. 4. Select /CH on the Remote Power Control table. 5. Select Graceful Shutdown and Power Off from the Actions drop-down list. 6. Click Save. How to Power Off the System Using the CLI 1. Log in to the ILOM CLI interface. 2. Type the following command: -> stop -f /CH 2-10 Sun Blade 6000 Modular System Service Manual December 2010

25 This command powers off the server with a non-graceful shutdown of the host operating systems. Caution Using the -f option could result in data loss unless the host operating systems have already been shut down at the server module level. For instructions on powering on the system after service is complete, see Section 3.4, Powering on the System After Service on page Chapter 2 Preparing for Service 2-11

26 2.5 Required Installation Tools and ESD You need the following tools and supplies to service the Sun Blade 6000 modular system: No. 2 Phillips screwdriver (for removing the midplane or fan connector boards) M5 hex nut driver (for removing internal power cables) Antistatic wrist strap Antistatic mat See the following procedure for instructions on attaching the antistatic wrist strap. How to Attach the Antistatic Wrist Strap Use the following procedure to prevent static damage when you are accessing the server modules and server module options. Caution Printed circuit boards and hard disk drives contain electronic components that are extremely sensitive to static electricity. Ordinary amounts of static from your clothes or the work environment can destroy components. Do not touch the components themselves or any metal parts without taking proper antistatic precautions. 1. Use an antistatic mat or similar surface. When installing an option in the server module, such as DIMMs or disk drives, place those static-sensitive parts on an antistatic surface. You can use the following items as an antistatic surface: The bag used to wrap an Oracle part The shipping container used to package an Oracle part Oracle electrostatic discharge (ESD) mat (part number , available through your Oracle sales representative) Disposable ESD mat (shipped with some Oracle parts and options) 2. Attach an antistatic wrist strap. Attach one end of the wrist strap to the chassis sheet metal, and the other end to your wrist. Refer to the instructions that come with the strap. 3. Detach both ends of the strap when you are finished installing the parts Sun Blade 6000 Modular System Service Manual December 2010

27 CHAPTER 3 Replacing System Components This chapter contains information and procedures for servicing the Sun Blade 6000 modular system hardware, including component removal and replacement procedures. The following topics are covered in this chapter: Section 3.1, Customer-Replaceable Units on page 3-2 Section 3.2, Field-Replaceable Units on page 3-28 Section 3.3, Servicetool FRU Update Procedure on page 3-38 Section 3.4, Powering on the System After Service on page

28 3.1 Customer-Replaceable Units The following procedures describe how to replace customer-replaceable unit (CRU) components: Section 3.1.1, Replacing Power Supplies on page 3-2 Section 3.1.2, Replacing the Front Fan Modules on page 3-5 Section 3.1.3, Replacing Server or Storage Modules on page 3-8 Section 3.1.4, Replacing the Front Indicator Module on page 3-11 Section 3.1.5, Replacing NEMs on page 3-13 Section 3.1.6, Replacing PCIe EMs on page 3-16 Section 3.1.7, Replacing the Rear Fan Modules on page 3-20 Section 3.1.8, Replacing the Fan Board on page Replacing Power Supplies You must replace a power supply if the module fails. However, do not leave a power supply slot vacant. Power supplies have integrated fans that continue to supply cooling air even when the power supply is off. You can remove and replace a power supply from a powered-off system or from a powered-on system using a hot-swap procedure. Note Make sure that you install the power supply supported for the Sun Blade 6000 chassis that you have. See Section 2.1, Identifying Chassis Models and Replacement Components on page 2-1 for more information. Use the following procedures to replace a power supply: How to Remove a Power Supply Module on page 3-2 How to Install a Power Supply Module on page 3-4 How to Remove a Power Supply Module FIGURE 3-1 shows how to remove the power supplies. 3-2 Sun Blade 6000 Modular System Service Manual December 2010

29 FIGURE 3-1 Removing a Power Supply Note You must replace the power supply that you removed within one minute in order to ensure proper system operation. 1. In the front of the chassis, locate the power supply to be removed. 2. Press and hold the green button on top of the power supply handle frame. 3. Rotate the power supply handle down and away from the power supply. 4. Pull the power supply out of the chassis approximately 100 cm. 5. While supporting the bottom of the power supply module with both hands, pull the power supply all the way out of the chassis. Note Only remove one power supply at a time. One power supply must be in the system at all times to supply power to the system. Chapter 3 Replacing System Components 3-3

30 How to Install a Power Supply Module Note When removing a power supply from the system, wait for all LED indicators on the power supply to be extinguished before reinstalling the power supply. If you do not follow this procedure, the system might log a false power supply fault condition. FIGURE 3-2 shows how to install the power supplies. FIGURE 3-2 Installing a Power Supply 1. On the front of the system chassis, locate a vacant power supply slot. 2. Ensure that the power supply unit is facing up and the five LED holes appear on the left when installing the power supply module. 3. Pull the power supply handle out and away from the power supply. 4. Align the power supply unit with the power supply slot. 5. In a smooth motion, slide the power supply into the power supply slot until the unit engages with the internal connectors. 6. Push the power supply handle back toward the power supply until it clicks into place. 3-4 Sun Blade 6000 Modular System Service Manual December 2010

31 3.1.2 Replacing the Front Fan Modules You must replace a front fan module if the module fails. A front fan module is located within each of the power supply modules, so you must remove the power supply associated with the fan module before replacing the module. Do not leave a power supply slot vacant. Power supplies have integrated fans that continue to supply cooling air even when the power supply is off. You can remove and replace a power supply from a powered-off system or from a powered-on system using a hot-swap procedure. Note When removing a power supply from the system, wait for all LED indicators on the power supply to be extinguished before reinstalling the power supply. If you do not follow this procedure, the system might log a false power supply fault condition. Use the following procedures to replace the front fan modules: How to Remove a Front Fan Module on page 3-5 How to Install a Front Fan Module on page 3-7 How to Remove a Front Fan Module FIGURE 3-3 shows how to remove the front fan modules. Chapter 3 Replacing System Components 3-5

32 FIGURE 3-3 Removing a Fan Module Note You must replace the power supply that you removed within one minute in order to ensure proper system operation. 1. Follow the instructions for removing a power supply in Replacing Power Supplies on page 3-2. You only need to pull the power supply out of the system far enough to access the fan. You do not need to remove it completely from the system. 3-6 Sun Blade 6000 Modular System Service Manual December 2010

33 Caution The fan continues spinning for a moment after the power supply is disconnected. Be careful not to insert a finger or other object into the fan while it is still spinning. 2. With one hand, squeeze together the two latches on top of the fan module and pull the module upward. 3. Grasp the module with the other hand as soon as possible. How to Install a Front Fan Module FIGURE 3-4 shows how to install the front fan modules. FIGURE 3-4 Installing a Front Fan Module 1. Insert the new fan module into the front of the power supply. 2. Follow the instructions for installing a power supply in How to Install a Power Supply Module on page 3-4. Chapter 3 Replacing System Components 3-7

34 3.1.3 Replacing Server or Storage Modules You can remove and replace a server or storage module (blade module) from a powered-off system or from a powered-on system using hot-plug procedures. Caution Do not operate system with empty slots. Always insert a filler into an empty slot to reduce the possibility of a module overtemp shut down. Use the following procedures to replace the server or storage modules: How to Remove a Server or Storage Module on page 3-8 How to Install a Server or Storage Module on page 3-9 How to Remove a Server or Storage Module 1. Initiate an orderly shutdown of the operating system. Refer to the documentation for your blade module for detailed instructions. 2. Power down the blade module to standby power. Refer to the documentation associated with the blade module for detailed instructions. 3. Press the latches on the ejectors at the top and bottom of the blade module. 4. Simultaneously swing out both ejector latches (raise the top ejector and lower the bottom ejector). 5. Use the ejector latches to start pulling the blade module out of its slot in the chassis. Caution Do not use the ejector latches to carry the blade module. The ejector latches should only be used to unseat the blade module from the chassis. The latches are not designed to support the weight of the blade module. 6. Use your hands to finish pulling the blade module out of the chassis, supporting the weight of the blade module from the bottom. 7. Install a filler panel into the blade module slot if the blade module is out of the chassis for longer than one minute. 3-8 Sun Blade 6000 Modular System Service Manual December 2010

35 How to Install a Server or Storage Module 1. Remove the filler panel, if necessary. FIGURE 3-5 Removing a Filler Panel 2. Rotate the lower ejector lever on the filler panel downward, and pull the filler panel out of the chassis. Note Do not discard the filler panel. You must reinstall the filler panel if the blade module needs to be serviced or removed at a later time. Chapter 3 Replacing System Components 3-9

36 FIGURE 3-6 shows how to install a blade module. FIGURE 3-6 Installing a Blade Module 3. Align the blade module with the vacant blade module slot. Ensure that the: Blade module indicator lights on the front panel are facing toward you. Blade module ejector levers are fully opened. Blade module ejectors are on the right side of the module. 4. Slide the blade module into the vacant blade module chassis slot. 5. As the ejectors make contact with the chassis, the bottom lever starts to rise, and the top lever starts to lower. 6. Simultaneously close the ejector levers at the top and bottom of the blade module (lower the top ejector and raise the bottom ejector) until the levers clamp in place against the front of the blade module. As the ejector levers are closed, the blade module moves back to engage with the chassis Sun Blade 6000 Modular System Service Manual December 2010

37 7. Verify that the blade module is powered on and operational. After you insert a blade module into a powered-on system, the blade module s internal service processor (SP) is brought online automatically and the blade module is powered on by default Replacing the Front Indicator Module The front indicator module (FIM) is located in the front of the chassis. You can remove and replace a FIM from a powered-off system or from a powered-on system using hot-plug procedures. Use the following procedures to replace the front indicator module: How to Remove the Front Indicator Module on page 3-11 How to Install the Front Indicator Module on page 3-12 How to Remove the Front Indicator Module 1. Remove power supply 0 from the chassis. See Section 3.1.1, Replacing Power Supplies on page 3-2. FIGURE 3-7 shows how to remove the FIM. Chapter 3 Replacing System Components 3-11

38 FIGURE 3-7 Removing the FIM 2. Push back the power supply air baffle. 3. Press the button on the right side of the front indicator module (FIM). 4. Remove the FIM from the chassis. How to Install the Front Indicator Module 1. Align the FIM with the FIM slot. Make sure that the release button is on the right side of the module. FIGURE 3-8 shows how to install the FIM Sun Blade 6000 Modular System Service Manual December 2010

39 FIGURE 3-8 Installing the FIM 2. Slide the FIM into the FIM slot. 3. Install power supply 0 into the chassis. See Section 3.1.1, Replacing Power Supplies on page Replacing NEMs If a network express module (NEM) fails, you must replace a NEM. You can remove and replace a NEM from a powered-on system using a hot-swap operation, as well as from a powered-off system. Caution If you are not immediately replacing the NEM, install an NEM filler panel to maintain proper system cooling. Use the following procedures to replace a NEM: How to Remove a NEM on page 3-14 How to Install a NEM on page 3-14 Chapter 3 Replacing System Components 3-13

40 How to Remove a NEM FIGURE 3-9 shows how to remove a NEM. FIGURE 3-9 Removing a NEM 1. In the rear of the chassis, locate the NEM that you want to remove. 2. Remove all cables from the NEM. 3. Squeeze the ejector buttons on both the right and left ejector levers. 4. To unlatch the NEM from the chassis, open the ejector levers by extending them outward. 5. Holding the opened ejector levers, pull the NEM toward you until you can pull the rest of the module out by hand. How to Install a NEM FIGURE 3-10 shows how to install a NEM Sun Blade 6000 Modular System Service Manual December 2010

41 FIGURE 3-10 Installing a NEM 1. Align the NEM with the vacant NEM slot. Ensure that: The RJ-45 port connectors of the NEM are facing you. The NEM ejector levers are fully opened. The NEM ejector levers are on the top of the module. 2. Slide the NEM into the vacant NEM chassis slot until you feel it stop. 3. Complete the installation by closing the ejector levers to secure the NEM in the chassis. Chapter 3 Replacing System Components 3-15

42 3.1.6 Replacing PCIe EMs A pair of PCIe Express Modules (PCIe EMs) is assigned to each server module in the chassis. All PCIe EMs connect to the chassis midplane. The midplane provides an interconnect between the server modules and the PCIe EMs installed in the chassis. The midplane connections between the server module and the PCIe EM are logically defined by the midplane as shown in FIGURE FIGURE 3-11 PCIe EM to Server Module Correspondence If the PCIe Express Module (PCIe EM) fails or if you choose to change the I/O configuration, you must replace the PCIe EM. You can remove and replace a PCIe EM from a powered-on system using a hot-plug procedure, as well as from a powered-off system. Use the following procedures to replace a PCIe EM: How to Remove a PCIe EM on page 3-17 How to Install a PCIe EM on page Sun Blade 6000 Modular System Service Manual December 2010

43 Caution If you are not immediately replacing the PCIe EM, install a PCIe EM filler panel to maintian proper system operation. How to Remove a PCIe EM Note Check your server module operating system documentation to determine if software commands are needed for the system to recognize PCIe EM removal or installation. FIGURE 3-12 shows how to remove the PCIe EMs. FIGURE 3-12 Removing a PCIe EM 1. In the rear of the chassis, locate the PCIe EM that you want to remove. If the amber Attention indicator on a PCIe EM is lit, this indicates a problem with the PCIe EM, and you might choose to replace it. Otherwise, if you want to change the I/O configuration, you can choose another PCIe EM to replace the faulty one. Chapter 3 Replacing System Components 3-17

44 2. If you are hot-plugging the PCIe EM, prepare the operating system. You must prepare for PCIe EM removal through the associated server module operating system. This can be accomplished with either OS commands or the Attention button. Refer to the server module (blade), operating system, or PCIe documentation for more information. 3. Verify that the PCIe EM Power indicator is Off. If you have successfully prepared the PCIe EM for a hot removal, the Power indicator is off. 4. Place one finger on top of ejector-lever handle. 5. Pull the ejector lever toward you and pull out the PCIe EM until you can remove the rest of the module by hand. How to Install a PCIe EM FIGURE 3-13 shows how to install a PCIe EM. FIGURE 3-13 Installing a PCIe EM 3-18 Sun Blade 6000 Modular System Service Manual December 2010

45 1. Do one of the following, depending on the type of PCIe EM that you are installing: If you are not installing a Gigabit Ethernet PCIe EM, go to Step 2. If you are installing a Gigabit Ethernet PCIe EM, continue with the following procedure: a. Press the Release button on the side of the GbE PCIe EM to release its cover, then remove the cover. b. Locate the PCIe EM MAC address, which is visible on the PCIe EM with its cover removed. c. Record the MAC address and the slot number into which you are about to install the GbE PCIe EM. Keep the data for future reference. d. Replace the GbE PCIe EM cover by sliding the cover into place. 2. Align the PCIe EM with the vacant PCIe EM slot. Ensure that the indicator lights on the front panel of the PCIe EM are facing toward you and that the PCIe EM ejector lever on the bottom is fully opened. 3. Slide the PCIe EM into the vacant PCIe EM chassis slot. The ejector lever starts to pop up as the module engages with the system slot. 4. Complete the installation by closing the ejector lever to secure the PCIe EM in the chassis. 5. Notify the operating system that a new PCIe EM is installed. Refer to the PCIe EM documentation for more information. 6. Verify that the PCIe EM Power indicator is On. If the hot-plug procedure was successful, the green Power indicator illuminates. Chapter 3 Replacing System Components 3-19

46 3.1.7 Replacing the Rear Fan Modules You must replace a rear fan module if the module fails. You can remove and replace a rear fan from a powered-off system or from a powered-on system using a hot-swap procedure. Note Make sure that you install the rear fan module supported for the Sun Blade 6000 chassis that you have. See Section 2.1, Identifying Chassis Models and Replacement Components on page 2-1 for more information. Use the following procedures to replace a rear fan module: How to Remove a Rear Fan Module on page 3-20 How to Install a Rear Fan Module on page 3-21 How to Remove a Rear Fan Module FIGURE 3-14 shows how to remove a rear fan module. FIGURE 3-14 Removing a Rear Fan Module 3-20 Sun Blade 6000 Modular System Service Manual December 2010

47 1. At the rear of the chassis, locate the fan module that you want to remove. 2. Press and hold the green button on the fan handle. 3. Pull out the rear fan module in a smooth motion until it is free from the chassis. How to Install a Rear Fan Module FIGURE 3-15 shows how to install rear fan modules. FIGURE 3-15 Installing a Fan Module 1. In the back of the system chassis, locate a vacant fan module slot. 2. Using both hands, align the fan module with the vacant slot. Make sure that the green button on the handle of the fan module is at the top of the fan. 3. Slide the fan module into the vacant slot until the front panel meets the chassis. After you install the rear fan module, the chassis management module (CMM) automatically detects the new module. Since there are no presence sensors, and no FRUID on fan modules, replacing the fan requires you to clear the fault in ILOM. 4. Clear the fault in the ILOM web interface or CLI. To clear the fault using the web interface: a. Login to the ILOM web interface. The Chassis View page appears. Chapter 3 Replacing System Components 3-21

48 b. Navigate to System Information -> Fault Management. c. In the Fault Management page, enable the radio button next to the faulted component, then select Clear Fault. To clear the fault using the CLI: a. Log into the ILOM CLI. b. Click on CMM in the left navigation panel. c. Type the following command: ->set component_path clear_fault_action=true For example, to clear the fault for Fan 0. ->set /CH/FM0 clear_fault_action=true Replacing the Fan Board You must replace a fan board if the board fails. You can remove and replace a fan board from a powered-off system. Use the following procedures to replace a fan board: How to Remove a Fan Board on page 3-23 How to Install a Fan Board on page Sun Blade 6000 Modular System Service Manual December 2010

49 How to Remove a Fan Board FIGURE 3-16 shows how to remove the fan boards. FIGURE 3-16 Removing a Fan Board 1. Power off the chassis. See Section 2.4, Powering Off the System Before Service on page Remove the two rear fan modules that are in front of the fan board that you want to remove. See Replacing the Rear Fan Modules on page Loosen the captive screw on the fan board until you can pull the top of the controller forward. Then lift the board off the chassis. This screw needs to be fully unscrewed because the screw disengages the fan controller connector from the midplane. Chapter 3 Replacing System Components 3-23

50 How to Install a Fan Board FIGURE 3-17 shows how to install the fan boards. FIGURE 3-17 Installing a Fan Board 1. Place the bottom notch on the fan board into the hole in the chassis 2. Push the top captive screw into place and tighten it. 3. Replace the two rear fan modules that you removed. See Section 3.1.7, Replacing the Rear Fan Modules on page Power on the chassis. See Section 3.4, Powering on the System After Service on page Sun Blade 6000 Modular System Service Manual December 2010

51 3.1.9 Replacing the CMM You must replace a chassis management module (CMM) if the module fails. You can remove and replace a CMM from a powered-off system or from a powered-on system using a hot-swap procedure. The chassis will be unmanageable while you replace the CMM hardware. Note Make sure that you install the CMM supported for your chassis. See Section 2.1, Identifying Chassis Models and Replacement Components on page 2-1 for more information. Use the following procedures to replace the CMM: How to Remove the CMM on page 3-25 How to Install the CMM on page 3-26 How to Remove the CMM FIGURE 3-18 shows how to remove the CMM. FIGURE 3-18 Removing the CMM Chapter 3 Replacing System Components 3-25

52 1. In the rear of the chassis, locate the CMM. 2. Remove any cables attached to the CMM. 3. Squeeze together and hold the ejector button on the module ejector lever. 4. To unlatch the module from the chassis, pull the ejector lever away from the module and up. 5. Pull the ejector lever toward you until you are able to pull the rest of the module out by hand. Note The system fans run at 100% speed when the CMM is removed from the system, as there is no fan management while the CMM is out of the system. How to Install the CMM FIGURE 3-19 shows how to install the CMM Sun Blade 6000 Modular System Service Manual December 2010

53 FIGURE 3-19 Installing the CMM 1. Align the CMM with the CMM slot. Ensure that the CMM indicator lights on the front panel are on the top of the module and facing you. 2. Slide the CMM into the vacant CMM chassis slot. Ensure that the CMM engages with the system chassis guidance system. 3. Complete the installation by closing the ejector lever to secure the CMM in the chassis. After you physically install a CMM in a powered-on system, the CMM automatically begins its initialization. If you have installed the CMM in a powered-off system, the CMM goes through the following sequence of operations: a. CMM receives 3-volt auxiliary standby power and comes out of reset. b. While the CMM boots up, the green OK indicator on the CMM is set to the Slow Blink state. Chapter 3 Replacing System Components 3-27

54 c. When the CMM has booted, the CMM sets the green OK indicator on the front indicator (FIM) and chassis management module (CMM) to the Standby Blink state, indicating that the chassis is now ready for full power-on. 3.2 Field-Replaceable Units The following procedures should only be performed by Oracle Service personnel Replacing the Midplane The midplane is not a hot-swap or hot-plug component. You must power off the chassis before replacing this component. Note Make sure that you install the midplane supported for your chassis. See Section 2.1, Identifying Chassis Models and Replacement Components on page 2-1 for more information. Use the following procedures to replace the midplane: How to Remove the Midplane on page 3-29 How to Install the Midplane on page Sun Blade 6000 Modular System Service Manual December 2010

55 How to Remove the Midplane Use the following procedure to remove the midplane. 1. Power off the chassis. See Section 2.4, Powering Off the System Before Service on page Remove all the modules or filler panels that are installed in the chassis. See the procedures in Section 3.1, Customer-Replaceable Units on page Remove the air duct by unfastening the snaps from underneath the power supply cage and tilting the front of the air duct downward to free it from the chassis. FIGURE 3-20 shows how to remove the air duct. FIGURE 3-20 Removing the Air Duct Chapter 3 Replacing System Components 3-29

56 FIGURE 3-21 shows how to remove the midplane. FIGURE 3-21 Removing the Midplane 4. Loosen the 13 captive screws that attach the midplane to the back of the chassis. Caution Do not touch the connectors on the midplane. This could result in damaged or bent connector pins. 5. Pull the midplane away from the chassis guidepins, then downward to clear the power supply cage. 6. Tilt the midplane slightly to the side to remove it from the chassis Sun Blade 6000 Modular System Service Manual December 2010

57 How to Install the Midplane FIGURE 3-22 shows how to install the midplane. FIGURE 3-22 Installing the Midplane 1. Tilt the midplane to the side slightly to fit it into the chassis. 2. When you reach the back wall of the chassis, align the midplane so that it is flush with the back of the chassis, and lift it so that the guidepin holes are aligned with the chassis guidepins. Caution Electrical short hazard caution - Make sure the side gaskets are touching the edge of the midplane and are not crushed. If during the installation, the airflow gasket is damaged or loosened in any way, then complete removal of the gasket is recommended so that an electrical short hazard is not present. 3. Push the midplane backwards slightly to engage the chassis guidepins. 4. Fasten the 13 screws to secure the midplane to the chassis. Chapter 3 Replacing System Components 3-31

58 FIGURE 3-23 shows how to install the air duct. FIGURE 3-23 Installing the Air Duct 5. Replace the air duct. a. Align the air duct so that the fastening snaps are facing away from the inside of the chassis. a. Place the back edge of the air duct in the slot on the back of the chassis. b. Rotate the air duct upward until the snaps fasten into place at the back of the power supply cage. 6. Replace all of the components that were removed from the chassis before installing the midplane. See the procedures in Section 3.1, Customer-Replaceable Units on page Power on the chassis. See Section 3.4, Powering on the System After Service on page Sun Blade 6000 Modular System Service Manual December 2010

59 How to Update FRU Information After Replacing the Midplane Caution The SunService account is for the use of Orace service representatives only. Do not use the SunService account unless you are instructed to do by Oracle service. 1. Use SSH to log into the SunService account. Obtain login information from Oracle service. 2. After replacing the midplane, issue the following command: # servicetool --board_replaced=midplane The following output appears: Servicetool is going to update the midplane FRU with product and chassis information collected from the removed midplane. The following preconditions must be true for this to work: * The new midplane must be installed. * The service processor must not have been replaced with the midplane. * The service processor firmware must not have been upgraded prior to the midplane replacement; do firmware upgrades after component swaps! Do you want to continue (y n)? y 3. Type y. 4. Reboot the CMM so that CMM recognizes the new serial number. 5. Make sure that the midplane FRU number has been updated. a. Log into the CMM ILOM. b. Type the following command: -> show /CH c. Verify that the midplane serial number has been updated. Chapter 3 Replacing System Components 3-33

60 Note If the serial number data isn t cached the servicetool --board_replaced= midplane" method might not work to update the serial number. If the midplane serial number was not updated, use these commands to update the chassis serial numbers manually: a. Run the following command: # servicetool --fru_chassis_serial_number The following output appears: Servicetool is going to update the FRU chassis serial number Do you want to continue (y n)? y b. Type y to continue. The following output appears: When entering values, do not use quotes; If you require embedded quotes, escape them with three backslashes; e.g. \\' What is the new chassis serial number? XXXXXXXXXX Where XXXXXXXXXX represents is the new serial number for the chassis. c. Enter the new serial number. The following output appears: The chassis serial number has been updated. The new serial number is: "XXXXXXXXXX" Updating FRUs... done d. Reboot the CMM so that the CMM recognizes the new serial number. Note The fault LED stays lit until you disable escalation mode and reset the CMM again Sun Blade 6000 Modular System Service Manual December 2010

61 3.2.2 Replacing the Internal Power Cables The power cables are not a hot-swappable or hot-pluggable components. You must power off the chassis before replacing them. Use the following procedures to replace the power cables: How to Remove an Internal Power Cable on page 3-35 How to Install an Internal Power Cable on page 3-37 How to Remove an Internal Power Cable 1. Power off the chassis. See Section 2.4, Powering Off the System Before Service on page Remove all the modules or filler panels that are installed in the chassis. See the procedures in Section 3.1, Customer-Replaceable Units on page Remove the midplane from the chassis. See Section 3.2.1, Replacing the Midplane on page FIGURE 3-24 shows how to remove the power cables. Chapter 3 Replacing System Components 3-35

62 FIGURE 3-24 Removing the Internal Power Cables 4. Locate the power cable that you need to remove. 5. Remove the retention clip and grounding clip holding the cable in the chassis housing. 6. Pull the cable downward to release it from the chassis cable housing. 7. Unfasten the cable from the metal tabs that secure it to the chassis wall. 8. Unscrew the small panel on the right rear side of the chassis. 9. Remove the two terminal clips that correspond to the power cable that you are removing Sun Blade 6000 Modular System Service Manual December 2010

63 10. Using a nut driver, remove the chassis ground connectors that correspond to the power cable that you are removing. 11. Remove the cable from the chassis. How to Install an Internal Power Cable FIGURE 3-25 shows how to install the power cables. FIGURE 3-25 Installing a Power Cable 1. Replace the connector end of the cable in the chassis cable housing. 2. Replace the grounding clip and retention clip. 3. Route the cable through the tabs on the back wall of the chassis. 4. Route the end wires to the cable housing on the right rear of the chassis. Chapter 3 Replacing System Components 3-37