Slide 0 Welcome to the Support and Maintenance chapter of the ETERNUS DX90 S2 web based training. 1 This module introduces support and maintenance related operations and procedures for the ETERNUS DX60 S2. Some of the sections are focusing on those functions of the built-in ETERNUS Web GUI that are needed for typical maintenance procedures, whereas some sections describe useful functions for daily technical support work. 2 This first chapter focuses on those functions that are typically needed for standard Maintenance Operations. These functions can be found from the Device Status menu. 3 This screen we have already seen couple of times in the previous chapters. On this screen you can see the overall status of the Controller Enclosure and of the Drive Enclosure, if there is one attached to the system. Should any component in the Controller or Drive Enclosure have an error, it would be indicated here. On this screen it is possible to confirm the status of each device and component by clicking on the item of interest, after which all the related information will be shown. The Maintenance Operations are conducted from this screen, for each device or component of the system; functions like Replace or Force Disable are shown on the bottom right corner of the screen. The available functions depend on the component being selected and its current status. 4 As already mentioned in the Hardware and Software Installation chapter, the ETERNUS DX60 S2 can indicate need for preventive maintenance by changing the system status from Normal to Warning and showing which component is in need of preventive maintenance. This function is applicable to all those components that are available as spare parts. Preventive maintenance aims to prevent component failures before they occur by suggesting a parts replacement prior to the failure. The system mode must be set to Maintenance before starting preventive maintenance actions. 5 With the Force Disable function, you can force a detachment of a component, in other words force the system to a degraded state. Force Disable can be used to detach a component which for example has not reached a normal status after a service procedure. Force Disable can be executed on any component regardless of its current status. 6 Opposite to what we can do with the Force Disable, with the Force Enable we can attach a component to the system by forcing the component to reboot.
For example, if during a service procedure we have removed from the system a component due to a failure of another component, the Force Reboot can be used reattach the component. 7 If you want to permanently remove a disk from the system due to a defect or if you for example have unused disk or disks in one ETERNUS system and you want to use them in another ETERNUS system, the disks can detached from the system using the Reduce Disk function. Please note that functional disks must not be removed by simply pulling them out, this function must always be used when removing a disk from the system. 8 To physically locate the system or its particular module you can enable the Identify LED. On the Controller Module and Drive Enclosure the LED is located in the Operation Panel on the left hand side of the front panel. The Identity LEDs of the CMs and PSUs are visible in the backside of the unit. When activated this LED is blinking in blue. 9 The RAID Group Status screen shows the list of RAID Groups and the detailed information of each registered RAID Group. This function provides System Administrators and Maintenance Engineers an overview of the RAID Groups and the Logical Volumes assigned to the RAID Groups. Also an ongoing Rebuild or Copy Back and Expand progress is shown here. 10 On the Volume Status screen you can see the status of each Volume registered in the device; if a Volume is being currently formatted or migrated it is shown here. 11 Next we will have a look at the Maintenance Utilities which are available under the menu Maintenance. 12 This function sets the system to a Maintenance mode, which means that remote reporting is deactivated, Sense Information is ignored and the status LEDs are disabled. This enables maintenance work on the system without generating any unnecessary error messages. It is very important to set the system back to "Normal" mode after the maintenance procedures are finished, in the last chapter of this module we will see how to do that. 13 The Add Drive Enclosure function of the Hardware Maintenance menu provides the possibility to add a Drive Enclosure to the system without having to switch the system off. This functionality is naturally only available with the 3.5 inch Controller Module. 14
The Remove Drive Enclosure is obviously the opposite function to adding a Drive Enclosure. A prerequisite for a removal of a Drive Enclosure is that all disks in it are unused, meaning, not allocated to a RAID Group. 15 If the system has only one Controller Module fitted, a second CM can be added with the Add Controller Module function. If there is Drive Enclosure attached to the Controller Enclosure, it makes obviously sense to fit at the same time a second SAS Expansion Module to the DE d-e]. In this case both the Controller Module and the Expansion Module are added to the system in one go. 16 This function deletes all statistical information and sense data, meaning internal system data which has not been reported to the host. Always after having put the system in Maintenance Mode for maintenance procedures that might cause the system to generate error notifications, it is recommended to delete the sense data. 17 In this chapter we will have a look at the different options the ETERNUS DX60 S2 offers to view various log information of the system and how to export this information for further analysis and reporting. 18 Through the Event Log menu you can access the event log of the system. The event log can hold up to 400 events. When the system log reaches this limit the older events are automatically deleted as new events are recorded. There are different types of log instances - Information, Warnings, Errors and Maintenance related - like the ones that we see here: creation or deletion of RAID Groups, system status related and component failures. Individual log entries can also be deleted here. The Event Log is held separate from the internal device logs, meaning that deleting an internal device log will not erase the Event Log and vice versa. 19 Panic Dump is a copy of the system operating memory content of the Controller Module at the time when a failure occurred. This memory dump can be used to analyze the cause of the failure. The dump file can be saved in various file sizes which enables for example use of floppies or providing a convenient file size for an email attachment. On this screen it is also possible to delete a specific Panic Dump. For each Controller Module a maximum of two Panic Dumps can be held, if more CM failures occur, the latest and the oldest will be held. In other words, from four occurred CM failures the first and the fourth are held. 20 From the Error Information menu it is possible to see an overview of the Disk Errors and Port Errors. This information can be very useful to identify degradation or early failure signs of the system components.
21 The Export Log function allows exporting the internal maintenance information log for recording the device maintenance history or for further problem analysis. As already mentioned by the Panic Dump, with "Log File Size" it is possible to define the maximum size of each block of log to be exported: the default size of 4.27 Megabytes produces the smallest amount but biggest size of files, whereas the other options produce smaller file sizes but respectively a larger number of files. Here one should select the file size which best suites the consequent handling of the log data. With this function it is also possible to delete chosen parts of the log information. 22 This function saves the configuration information of the ETERNUS DX60 S2 in a file that has an extension cfg. This file can be consequently used as a configuration file to copy a system configuration to other ETERNUS systems with identical hardware configuration. The file can also be used as a recovery file if problems are experienced after the configuration is changed, therefore it is recommended to export the configuration file before making changes to the system hardware configuration. 23 On this screen it is possible to see - and if wanted, to export - the G-List, or Grown Defect List, which is a list of disk related errors saved on each disk's disk management area. Please note that disk access from the Host must be prevented when extracting the G-List. 24 In this next chapter we will get ourselves familiar with some further system related utilities. There are utilities for system shutdown, shutdown and restart, and for initializing a system disk. The Cache utilities allow us to manage pinned data and to force a cache write back. Recovery utilities provide us several possibilities to recover the system if configuration related problems should occur. Setting up manually which Controller Module has the Master role can be done with the "Change Master CM" function. Furthermore, Diagnosis utilities provide the possibility to diagnose the disks and the RAID Groups. 25 For a remote shutdown and or restart in the case the front panel power switch has an error or the system hangs, this function provides the possibility to restart the system or to power it down. 26 Initialize system disk prepares a system disk - drive zero or one - for use in the ETERNUS DX60 S2. This function initializes all firmware archives, system logs and Panic dumps. After initializing a system disk it is necessary to register the active firmware. Failing to do that would for example cause problems with Controller Module maintenance and when attaching a second Controller Module or with Drive Enclosure expansion.
27 This function is for writing back or deleting "pinned data" generated in the device cache area. Pinned data is the data left in the cache due to unsuccessful write-back to the disk drive from the cache area. This can be caused for example when a RAID Group has not been accessible because of a power failure. 28 If the user data could not be written to the data disk from the cache area, this function forcibly writes back all the user data left in the cache area. 29 First of the Recovery utilities that we will have a look at is the Apply Configuration function. A system configuration can be restored from three different sources: From one of the three latest automatically saved configurations. From one of the four user defined backup configurations - following page shows how to back up the configuration - or from a configuration file which has been exported with the "Export Configuration" function as explained in the previous chapter. Choosing "Restore" as the apply mode would apply one of the above configurations, choosing "Initialize" would not only apply the above configuration but also reset the configuration insofar that the Volumes need to be formatted after the system has been once power cycled. 30 With the "Backup Configuration" function one can permanently save, or backup, a configuration. Like mentioned on the previous page, the configuration can be later restored should problems occur after a system configuration change - therefore it is recommended to always take copy of the system configuration before applying any changes to the configuration. 31 With this screen you can resolve a problem situation that is caused by a system problem which has occurred either during power down or power up. Backup Fail refers to a system status that has been caused by an error that occurred at power failure or during the power down sequence. Restore Fail is the status that results when the system is not able to complete the power on sequence because of an error at power on. In both cases, even if the cause of the error has been rectified, it may happen that the system does not come back to status "Normal". In such cases, clicking on Reset clears the Backup Fail or Restore Fail status. 32 If any abnormality is detected during the power on sequence, the device status is set to Not Ready and consequently no I/O can be received from the host. In this situation the user data saved in the system disk from the cache memory will not be restored back to the system cache memory. This function can be used to forcibly restore this user data to the system cache memory and thus avoid any loss of data. After clearing the cause of the problem, execute Force Restore by clicking on "Restore" and power off/on the device.
33 Should the very unlikely situation occur that an ETERNUS DX60 S2 with dual controller configuration looses one controller and panics at the same time, the system may end up with a status Machine Down. Consequently all Host paths are shut in order to save the Dirty Data, which refers to data inconsistency caused by data not having been synchronized between a RAID Group and the system cache. In order to avoid data loss, a system in Machine Down status cannot be switched off. Executing the "Reset Machine Down Recovery" function resets the status and allows the machine to be powered off and restarted again. 34 With this function it is possible to reboot the Controller Module or Modules fitted in the system, a reboot will be performed in 30 seconds after executing this function. 35 The Change Master CM function gives us the option to manually choose which of the two fitted Controller Modules has the role Master. To use this function, choose the CM which currently has the role Slave. The following page shows the roles of the CMs during maintenance operations and the possible ways to connect an ETERNUS and the management server or PC. 36 For the maintenance operations the management server or PC must be connected to the CM that has the Master role. This connection can be done either by direct connection, or by using a switch or a hub. Please note that only the networked connection is redundant. When connected over a network and the link to the Master CM goes down, the Slave takes over the Master role and its IP address automatically. A further option is to set an IP address for the Slave Controller Module in the "Set Network Environment" menu, in which case the Slave Controller takes over the functionality of the Master CM automatically if the LAN link to the Master CM goes down. 37 First of the two available Diagnosis utilities is the Disk Diagnosis. This function checks the state and the condition of the selected disks. Please note that the disks being diagnosed must not be in other use during the diagnosis, meaning there should be no Host access, migration, RAID Group change or similar in progress. After having chosen which disks to diagnose - the options are "All Disks", "Unused Disks", "Selected Disks" and "Selected Drive Enclosures" - the diagnose can be started. Depending on the number of disks and their capacities the diagnosis can take a longer or shorter time, after which the results are shown. 38 Similarly as the utility on the previous page is for diagnosing a disk or disks in the system, the RAID Group Diagnosis utility is for diagnosing the existing RAID Groups. This utility is especially useful when for example a disk multi-dead status has occurred after "All Controller Module Panic" and one wants to ensure the reliability of the RAID Group before restoring the data.
39 The Maintenance Procedures chapter introduces us to the Maintenance and Status LEDs, how to put the system in maintenance mode and what needs to be done before starting with Hot Maintenance procedures. 40 There is a General Status LED on the left hand side of the front panel. When the system is fully functional the status LED lights green. The other LED showing information about the system status is the general Fault LED. 41 In addition to the general Status, there are also LEDs on each module. These LEDs are named STS; when the LED is green the system is running normal. If a LED lights steady in orange then the module in question needs attention and typically needs to be replaced. 42 In the beginning of the chapter Maintenance Utilities we mentioned that before executing functions like add or remove Drive Enclosure the system needs to be set in Maintenance Mode. The purpose of the Maintenance Mode is to stop unnecessary external reporting You can verify the system status from the Device Status Menu 43 This brings us to the last chapter of this module. In this chapter we will have a look at the procedures and practices related to parts replacement, handling of system internal information and how to stop the Maintenance Mode and exit the GUI. 44 Let's see how to proceed when we want to do a hot replacement - meaning parts replacement without switching the system off - for the Controller Module one. First we choose Menu "Status" and Sub Menu "Device Status". Next we click on Controller Module 1, please notice the red symbol indicating a component error. The component can be replaced when the red symbol is shown, if the component has a status "Normal" or "Warning", it would be necessary first to "Force Disable" the component before proceeding. 45 After having enabled the hot replacement, the part can be replaced as per the instructions given in the Maintenance Manual. The table shows the typical times needed for a component replacement. Having replaced the component its status changes to show if the replacement was successful. 46 As mentioned on the previous page, after the replacement the component status must change from Maintenance to Normal. 47
If the replaced component was the only defective component in the system, the next step in the replacement procedure is to "Delete Sense Data". This prevents the error data that has potentially been generated by ETERNUS during the maintenance procedure from being sent out. 48 It is a good practice to collect the ETERNUS system internal logs after all maintenance work, as already mentioned in the chapter "Log and Dump Management", we can export the file in various file sizes. 49 In case of a Controller Module replacement, we recommend taking Panic Dump for analyzing the cause of failure. 50 As the last step of the replacement procedure it is important to stop the Maintenance Mode, this can be done from the "Start/End Maintenance" Menu. Always confirm from the "Device Status" Menu that the system returns back to "Normal" Mode. 51 After that we finish off our maintenance tasks by logging out of the GUI, however, before doing that we want to make absolutely sure that the system runs fully functional when we leave the customer site. To do that, we check the Device Status, the status of the RAID Groups and the status of all Volumes. We also check that the System LED on the left hand side of the front panel lights green. Only after that we log ourselves out of the system by clicking on "Logoff". 52 Here are useful links for reference documentation and additional information regarding the subjects covered in this training module. 53 That brings us to the end of this web based training module. Thank you for your attention.