Welcome to VPLEX VS6 FRU Replacements.

Transcription

1 Welcome to VPLEX VS6 FRU Replacements. Copyright 2016 EMC Corporation. All Rights Reserved. Published in the USA. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. The trademarks, logos, and service marks (collectively "Trademarks") appearing in this publication are the property of EMC Corporation and other parties. Nothing contained in this publication should be construed as granting any license or right to use any Trademark without the prior written permission of the party that owns the Trademark. EMC, EMC², the EMC logo, AccessAnywhere Access Logix, AdvantEdge, AlphaStor, AppSync ApplicationXtender, ArchiveXtender, Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, Avamar, Aveksa, Bus-Tech, Captiva, Catalog Solution, C-Clip, Celerra, Celerra Replicator, Centera, CenterStage, CentraStar, EMC CertTracker. CIO Connect, ClaimPack, ClaimsEditor, Claralert,cLARiiON, ClientPak, CloudArray, Codebook Correlation Technology, Common Information Model, Compuset, Compute Anywhere, Configuration Intelligence, Configuresoft, Connectrix, Constellation Computing, CoprHD, EMC ControlCenter, CopyCross, CopyPoint, CX, DataBridge, Data Protection Suite. Data Protection Advisor, DBClassify, DD Boost, Dantz, DatabaseXtender, Data Domain, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, DLS ECO, Document Sciences, Documentum, DR Anywhere, DSSD, ECS, elnput, E-Lab, Elastic Cloud Storage, Xaminer, Xtender, EMC Centera, EMC ControlCenter, EMC LifeLine, EMCTV, Enginuity, EPFM. eroom, Event Explorer, FAST, FarPoint, FirstPass, FLARE, FormWare, Geosynchrony, Global File Virtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, Illuminator, InfoArchive, InfoMover, Infoscape, Infra, InputAccel, InputAccel Express, Invista, Ionix, Isilon, ISIS,Kazeon, EMC LifeLine, Mainframe Appliance for Storage, Mainframe Data Library, Max Retriever, MCx, MediaStor, Metro, MetroPoint, MirrorView, Mozy, Multi-Band Deduplication,Navisphere, Netstorage, NetWitness, NetWorker, EMC OnCourse, OnRack, OpenScale, Petrocloud, PixTools, Powerlink, PowerPath, PowerSnap, ProSphere, ProtectEverywhere, ProtectPoint, EMC Proven, EMC Proven Professional, QuickScan, RAPIDPath, EMC RecoverPoint, Rainfinity, RepliCare, RepliStor, ResourcePak, Retrospect, RSA, the RSA logo, SafeLine, SAN Advisor, SAN Copy, SAN Manager, ScaleIO Smarts, Silver Trail, EMC Snap, SnapImage, SnapSure, SnapView, SourceOne, SRDF, EMC Storage Administrator, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix, Symmetrix DMX, Symmetrix VMAX, TimeFinder, TwinStrata, UltraFlex, UltraPoint, UltraScale, Unisphere, Universal Data Consistency, Vblock, VCE. Velocity, Viewlets, ViPR, Virtual Matrix, Virtual Matrix Architecture, Virtual Provisioning, Virtualize Everything, Compromise Nothing, Virtuent, VMAX, VMAXe, VNX, VNXe, Voyence, VPLEX, VSAM-Assist, VSAM I/O PLUS, VSET, VSPEX, Watch4net, WebXtender, xpression, xpresso, Xtrem, XtremCache, XtremSF, XtremSW, XtremIO, YottaYotta, Zero-Friction Enterprise Storage. 1

2 This course covers how to replace. Procedures are used to guide you through the steps along with videos showing the actual replacement of the units. 2

3 This module focuses on covering the preliminary FRU replacement information such as needed service software tools, the use of SSH and web server information. It will list the scripted and manual FRU procedures. 3

4 The latest VPLEX VS6 FRU procedures are always available in SolVe Desktop. There are CLI-guided and scripted procedures that guide you through replacements step by step including gathering health check and diagnostics information. The FRU replacements are typically started with the command VPlexadmin replace-fru and then select the faulted FRU from the replacement menu. On the other hand, there are some manual replacement procedures. No script guides you through the needed steps. The FRU manual lists all the steps which need to be executed manually. This manual process starts with disabling the call-home, gathering health information, and verifying the faulted unit for replacement. Another important area for FRU replacements is to know if the procedure is performed on an un-configured VPLEX VS6 Cluster, which means the configuration commands for EZ- Setup have not been run. An example could be that a faulted Management Module Control Station (MMCS) failed the EZ-Setup and it was determined to replace this MMCS. In this case the replacement procedure for an un-configured MMCS should be used. However, if an MMCS fails in a successfully configured VPLEX VS6 Cluster then the replacement procedure for a configured MMCS needs to be used. 4

5 The EMC VPLEX VS6 cabinet has a laptop tray in the front at the U20 location. The red Ethernet service cable connected to MMCS-A should be available to connect to the service laptop. The service laptop needs PuTTY version 0.60 or later for SSH connections, WinSCP or a similar tool for copying files to the VPLEX management module control stations, and a web browser and Flash Player version or higher for the VPLEX GUI. 5

6 When connected to the red Ethernet service cable, the laptop should automatically get an IP address from the DHCP server running on the MMCS. Use an SSH tool like PuTTY and configure it to the service IP address with the default port 22 for the connection type SSH. You can then save the session and open the session to the MMCS-A. 6

7 You can also connect to the same service port IP address with a browser using HTTP. In the FRU replacement screen, the commands are listed to start the scripted FRU replacement procedure. After you have logged into the MMCS-A and connected to the VPlexcli using SSH, you execute these commands. While going through the FRU Replacement script, there are entries for illustrations in the GUI. They can help with an easier selection and show how to remove and replace a FRU through illustrations. 7

8 With the VS6 the call-home feature should be used. A warning message is generated if a FRU is degraded. If a FRU is faulted, an error message is sent. The information in the call-home messages and the information gathered through troubleshooting come from the EMC support center and generate service calls. VPlexcli context information and health check commands can also be used to detect and verify faulted FRUs. In Unisphere for VPLEX, certain failures are displayed under the Monitoring tab in System Health. Service tasks can also be generated out of troubleshooting an issue. Some FRUs might not generate call homes or unique warning or error messaging. Performance issues can be one of the symptoms. Replacing an SFP or I/O Module might be a direct result from troubleshooting a customer SR. 8

9 When the faulted unit is selected in the replacement menu, a summary is displayed and should be checked against the FRU replacement call-home or service center information. Serial and part numbers can easily be found on the faulted unit. First we see pull-out tags for the engine chassis and the BBUs. Each fan module has its serial number and part number in the front. The management as well as the I/O modules have their serial and part number on the handle. The pull-out tag with serial and part number is from an InfiniBand switch. Other FRU components also have the S/N and part number easily available. 9

10 In the scripted procedures, VPLEX VS6 uses amber marker LEDs for engine, director, and unit to physically identifying the selected or faulted FRU to avoid errors. Here the engine marker and director marker can be seen. During a FRU replacement for a PSU, for example, the script also marks the PSU that was chosen in the menu and needs replacement. Another example for the use of amber marker LEDs is with a director fan. Fan replacements are a manual procedure, not script driven. The amber LED used for a faulted fan is enabled when the fault occurs and not activated with a replacement script. 10

11 This table of VPLEX VS6 FRUs shows which FRU is covered by a scripted, CLI-guided procedure, as well as which FRU has a procedure when the cluster has not yet been configured with EZ-Setup. Most of the FRUs are monitored and trigger a call-home. Some might not directly call home but can be identified through performance issues. 11

12 All FRU replacements in this table follow manual steps. As an example, IB switches are not directly monitored, but in case of a failure, this will show up in the internal communication statistic and IB-group presence. VPLEX monitors which constantly monitor software status, IO status, and hardware status, should detect the faulted IB connectivity. 12

13 This module focuses on the replacement of FRUs in the VPLEX VS6. Upon completion of this module you should be able to list all the VPLEX VS6 FRUs and perform their replacements. 13

14 This is a list of all the FRUs in the VPLEX VS6. All these FRUs have replacement procedures that must be followed when a replacement is needed. Perform this procedure only after a VPLEX cluster has been configured for operation by running the EZ-Setup utility. Do not run the procedure on a new cluster that has not yet been configured. FRUs listed here with an asterisk also have a procedure when the VPLEX cluster has not been configured. Make sure you use the correct replacement procedure. 14

15 Replacing a DIMM or memory module is completed from the front. You need to identify the engine, director, and the location of the faulted DIMM. The top view of the director shows four banks of DIMMs. Each bank has a total of 2 DIMMs, eight DIMMs per director in specific locations. Click the Launch button to view the DIMM replacement demonstration. 15

16 Two commands in the CLI context for DIMMs are shown here. The output of the first command displays the names of the DIMMs which also represent their location in the DIMM banks. The output of the second command shows the details per DIMM, its operational status, part and serial number. 16

17 Replacing the director Solid State Drive (SSD) is completed from the front. You need to identify the engine and director. The top view of the director shows the location of the SSD. The director supports the M.2 SSD - SMART Modular Technologies - with 60 mm in length. The thumb screw assembly holds the SSD in place. Click the Launch button to view the SSD replacement demonstration. 17

18 The command for the SSD in the VPlexcli context is shown here. The output lists the Solid State Drive (SSD) status, part number, and serial number. 18

19 The director chassis itself is a FRU. The director replacement chassis needs to be populated with the SSD, DIMM, BBUs, and Fans of the faulted director. The director FRU has two replacement procedures for the configured and un-configured state of the VPLEX cluster (after and before EZ-Setup). The difference between these procedures might only be very small but it is extremely important to use the correct procedure. In this demonstration the procedure for the configured director replacement is followed. Click the Launch button to view the director replacement demonstration. 19

20 A command for a director in the VPlexcli context is shown here. The output shows director attributes and director contexts dimms, fan-modules, internal-disks, io-modules, ports, and sfps. 20

21 Also serviced from the front of the VPLEX are the Battery Backup Units (BBU). Each engine has four: A0, A1, B0, and B1. The replacement script will mark the engine. The faulted BBU needs to be identified. Make sure you use the labeling on the engine chassis for selecting the right one. Click the Launch button to view the BBU replacement demonstration. 21

22 Two commands for the BBUs in the VPlexcli context are shown here. BBUs are in the engine context. The second command displays details of all the BBUs in one engine including battery charge and test, operational status, part and serial numbers. 22

23 Fan modules are serviced from the front. Each director has six fans named A0 to A5 respectively and B0 to B5. Labels on the engine chassis show the locations. If one fan needs to be replaced the director is still up and running. If a second fan on the same director fails or is pulled out, the director shuts down. The fan replacement is a manual procedure. Click the Launch button to view the director fan replacement demonstration. 23

24 Two commands for the Fan modules in the VPlexcli context are shown here. Fans are in the director/hardware context. The details for each fan include Fan status, marker LED, and operational status. 24

25 Power Supply Units (PSU) are serviced from the rear. Each engine has four PSUs named A0, A1 and respectively B0, B1. Labels on the engine chassis show the locations. If one PSU needs to be replaced the director is still up and running. Click the Launch button to view the PSU replacement demonstration. 25

26 Two commands for the PSUs in the VPlexcli context are shown here. PSUs are listed in the engine context. The details for each PSU include marker LED status, operational status, part and serial numbers. 26

27 The Base Modules (BM) are serviced from the rear. There are two de-populated BMs in an engine. The Base Module replacement is a manual procedure. Click the Launch button to view the base module replacement demonstration. 27

28 There are three Fibre Channel I/O Modules in each director for Front End, Back End, and WAN connectivity named A0, A1, A2 and respectively B0, B1, and B2. Replacement FC I/O modules come without the SFPs. SFPs are swapped from the faulted FC I/O module to the replacement module. Click the Launch button to view an FC I/O module replacement demonstration. 28

29 Two commands for displaying I/O modules in the VPlexcli context are shown here. I/O modules are listed in the director/hardware context. A director has three FC I/O modules and one InfiniBand I/O module (A3 or respectively B3). The details for the FC I/O modules include marker LED, operational status, part and serial numbers, and type. 29

30 There is one two-port InfiniBand I/O Module in each director for intra-cluster communication, named A3 and respectively B3. In a single-engine VPLEX cluster the two ports of one director are connected directly to the other director. In a dual- or quad-engine VPLEX cluster, all InfiniBand ports are connected to the InfiniBand switches. Click the Launch button to view the InfiniBand I/O module replacement demonstration. 30

31 Two commands for displaying I/O modules in the VPlexcli context are shown here. I/O modules are listed in the director/hardware context. A director has one InfiniBand I/O module, A3 or respectively B3. The details for the InfiniBand I/O modules include marker LED, operational status, part and serial numbers, slot number and type. 31

32 The FC I/O modules have SFPs in each port for the 16 Gb connectivity. The SFP replacement is a manual procedure. Click the Launch button to view an SFP replacement demonstration. 32

33 A command for the SFP information in the VPlexcli context is shown here. SFPs are in the director/hardware context. The details for each SFP include part number, serial number, rxpower, tx-power, and temperature. Values for current-rx-power and current-tx-power should not be less than 100. Power values lower than 100 generate a call home event. 33

34 InfiniBand switches are serviced from the rear. They are part of the configuration in dualand quad-engine VPLEX clusters and connect the intra-cluster communication redundantly. Each switch and one InfiniBand port of each director is part of an IB group. With one InfiniBand switch failure, the intra-cluster communication between the directors is still possible and VPLEX operations are still online. The InfiniBand switch replacement is a manual procedure. Click the Launch button to view the IB switch replacement demonstration. 34

35 In multi-engine VPLEX clusters engine 2, 3, and 4 have two Management Modules (MM), one per director. They are referenced to as mgmt-module-a and mgmt-module-b. The management module provides Ethernet management network connectivity for the cluster as well as serial and USB connection to each of the directors. Click the Launch button to view the management module replacement demonstration. 35

36 Two commands for the mgmt-modules in the VPlexcli context are shown here. The Management Modules (MM) as well as the Management Module Control Stations (MMCS) are in the same engine context. They are both referenced to as mgmt-modules, here mgmt-module-a and mgmtmodule-b. The details for the management module A in engine 2 include the module type, here management-module, the operational status, the part number (different from the MMCS), and serial number. 36

37 Two Management Module Control Stations (MMCSs) are in the base engine 1 of the VPLEX cluster. There is an MMCS-A and an MMCS-B. A service person typically connects to MMCS- A for any replacement procedures. However, if the MMCS-A needs to be serviced and no service port connection to MMCS-A is possible, MMCS-B is partially used for the replacement. The MMCS modules also provide Ethernet management network connectivity for the cluster. In the replacement video we will follow the VPLEX MMCS configured procedure, but remember there is also an MMCS replacement procedure for an un-configured VPLEX cluster. Click the Launch button to view the MMCS-A replacement demonstration. 37

38 Two commands for the MMCS in the VPlexcli context are shown here. MMCS and MM (Management Module) are in the same engine context. They are both referenced to as mgmt-modules, here mgmt-module-a and mgmtmodule-b. The details for the MMCS include the module type, here mmcs, the operational status, the part number (different from the MM), and serial number. 38

39 In this FRU replacement scenario both management module control stations need to be replaced. A possible scenario could be that it was determined that internal network issues resulted from two faulty MMCSs. The MMCS modules provide Ethernet management network connectivity for the cluster. The Dual MMCS replacement sequence will first replace MMCS-B, then replace, re-image and restore MMCS-A. The factory image on MMCS-B is used to re-image MMCS-A to the cluster version. After a successfully completed replacement of MMCS-A, MMCS-B is reimage and restored. Click the Launch button to view the dual MMCS replacement demonstration. 39

40 One MRJ21 cable assembly is connected with its keyed connector to the MMCS-A, another one to the MMCS-B. It provides connectivity to the customer management network (black cable), as well as the redundant internal management networks in dual and quad engine configurations. The lime-colored (A-side) cable is the.252 subnet within the VPLEX cluster. The purplecolored (B-side) cable is the.253 subnet. The MRJ21 cable assembly replacement follows a manual procedure. For further Ethernet cabling in dual and quad engine VPLEX VS6 clusters view the provided cabling animation. Click the Launch button to view the MRJ21 cable assembly replacement demonstration. 40

41 If an engine chassis replacement needs to be performed, the single engine VPLEX cluster would have a considerable data unavailability downtime. With dual and quad engine clusters, the replacement can be performed with continued production. There might be a reduction in performance. Shown is the engine chassis of the VPLEX VS6 the replacement chassis would come without the disk fillers which also have to be moved from the faulted engine chassis to the replacement chassis, just like both directors with BBUs and director fans, and all the I/O modules and MMs or MMCSs. The engine chassis has a configured and un-configured procedure. A transport lift is needed because of the weight of the 4-U VS6 engine. 41

42 The command for displaying an engine in the VPlexcli context shows engine attributes and contexts. Listed are among others the attributes for ip-seed, enclosure-id, the engine family, part number and serial number. The VPlexcli context for the engine lists battery-backup-units, directors, fans, mgmt-modules, and power-supplies. 42

43 The uninterruptible power supplies back up the InfiniBand switches. UPS-A backs up IB switch A while UPS-B backs up IB switch B. Just like the InfiniBand switches, UPS are only part of the VPLEX VS6 configurations in dual and quad engine clusters. The procedure follows manual steps. When replacing a faulty UPS, the other UPS and InfiniBand connections need to be functional. Commands that check these conditions are run during the procedure. After the replacement, the UPS status should be back to normal and the InfiniBand connections fully functional. 43

44 The commands for the UPS display the names, battery status, operational status, part and serial numbers. 44

45 In VPLEX Metro configuration the FC Com cable is used for the WAN connectivity. The connectivity is setup in redundant FC port groups. The command connectivity validate-wan-com checks for WAN COM connectivity. If there are any connections missing, it will list the faulty connections. The FC port groups will only show PARTIAL connectivity. When the faulty FC WAN cable(s) is replaced, the two FC port groups should show their expected connectivity as FULL. Be aware that the output may not show missing connectivity if the failure is intermittent. For further cabling details use the provided cabling demonstration. 45

46 The UPS serial cables connect the UPS with the MM of the second engine. They are used to monitor the UPS. If a cable has faulted or is not connected, the command shown here will display the operational status of the UPS as being removed. No other information on the UPS is available. For this process it is important to locate the right UPS - the peer needs to be operational. Both UPSs and IB switches should be online again after the UPS replacement. For the cabling in dual and quad engine VPLEX VS6 clusters, view the provided cabling demonstration. 46

47 AC power cables use the colors grey and black in the VPLEX VS6 cluster. Grey cables are used for the A-side connections, black cables for the B-side connections. There are three scenarios of possible AC power cable failures: first - the connection between the Power Distribution Unit (PDU) and the VPLEX PSUs; second - AC cables between the PDUs and UPSs; third - AC power cables between UPSs and InfiniBand switches. All cables should always be checked if they are seated completely and the bail is in place. If a cable needs to be replaced, follow the AC power cable manual replacement procedure. For the AC power cabling view the provided cabling demonstration. 47

48 Use the shell command VPlexPlatformHealthCheck. It pings all IP addresses in the cluster and identifies unreachable components as shown in this output example. The replacement manual has IP address tables for the dual and quad engine VPLEX cluster configurations to determine which internal cluster management cable needs to be replaced. Purple and lime cables are used in dual- and quad-engine VPLEX VS6 clusters. Use the provided cabling demonstration for more details. 48

49 InfiniBand cables are used in single engine VPLEX VS6 cluster to connect the two directors in the base engine. The InfiniBand backbone handle the intra-cluster communication. With dual- and quad-engine VPLEX VS6 cluster, the InfniBand port are connected to the InfiniBand switches. The output confirms that the cluster has full connectivity in respect to intra-cluster communication using InfiniBand. If a cable is faulty then the command output would show which cable is faulty. The faulty cable on a specific engine and director reports that it is Missing all expected connectivity. 49

50 This course covered the FRU procedures, access tools, and physical identification methods. The videos for each FRU showed in detail the software and physical replacement steps for the faulted units. The new VPLEX CLI context commands helped to identify and get the status of FRUs This concludes the training. Proceed to the course assessment on the next slide. 50

51 Do not record. Feedback Link will be inserted by Production Team. 51

52 52