Dell EMC. VxRack System FLEX Version Upgrade Guide Upgrade Guide

Transcription

1 Dell EMC VxRack System FLEX Version Upgrade Guide Upgrade Guide Document revision 1.3 October 2017

2 Revision history Date Document revision Description of changes October Minor copy edits September Changed publication title to reflect RCM version. Minor updates to procedures. September Updated guide title with the version number of ScaleIO. Updated guide with upgrading RAID card firmware and drivers on the ScaleIO VM and Red Hat nodes. Updated guide with upgrading and installing StorCLI content. June Initial release Revision history 2

3 Contents Introduction...4 Upgrade prerequisites...5 Prerequisite skills... 5 General prerequisites... 5 Prerequisites for ScaleIO... 5 Prerequisite tasks...5 Upgrade best practices and planning...7 Estimating the upgrade time...8 Upgrading Converged System releases...10 Upgrading VxRack Controller components...11 Upgrading VxRack System compute nodes...12 Upgrading BIOS on VxRack System compute nodes...12 Upgrading the VxRack System node BMC firmware Upgrading the VxRack System node network card driver Upgrading StorCLI for ESXi nodes Upgrading StorCLI on ScaleIO SVM and Red Hat nodes...16 Upgrading RAID card firmware and driver on ScaleIO VMs...17 Upgrading RAID card firmware and drivers on Red Hat nodes Upgrading the VMware vsphere Infrastructure Management components Upgrading the VMWare Hypervisor ESXi...24 Upgrading the VMware vsphere Hypervisor ESXi using the ISO image on VxRack Controller hosts...26 Upgrading the VMware vsphere Hypervisor ESXi using the ISO image on ScaleIO nodes Upgrading ScaleIO Upgrading the ScaleIO GUI Removing the EMC ScaleIO plug-in from VMware vcenter Registering the new ScaleIO plug-in...31 Deleting and uploading the OVA template Upgrading the Gateway Upgrading LIA...33 Upgrading the Secondary MDM...33 Upgrading the Primary MDM Upgrading the Tie-breaker Upgrading the SDS...34 Upgrading Red Hat Upgrading the SDC...35 Verifying the ScaleIO System upgrade...36 Monitoring component upgrades Contents

4 Introduction This document describes how to upgrade VxRack System FLEX to new software releases. These upgrade instructions apply to the following VxRack System releases: VxRack System release Based On VMware vsphere Hypervisor ESXi 6.0 When to use this guide Use this guide to incrementally upgrade from one VxRack System RCM release to another. From this release To this release On these systems VxRack System FLEX In this document, VxRack System FLEX will be referred to as a Converged System. Refer to the Glossary for terms, definitions, and acronyms that are related to Converged Systems. Introduction 4

5 Upgrade prerequisites Prerequisite skills This topic describes the skills required for performing this upgrade. To perform this upgrade, you should have the following knowledge and skills: Converged System concepts and terminology Troubleshooting Converged Systems Experience in VMware ESX and/or Linux system administration Understand virtual machine technology and datacenter operations Ensure that you understand the entire upgrade process before you begin the Converged System upgrade. Complete all of the planning and preparation requirements before you begin the upgrade. General system prerequisites This topic describes the general system prerequisites for performing an upgrade. Before starting the upgrade, you should have access to the following: Network access to the Converged Systems and the management systems Utilities for ping, traceroute, SSH, FTP server, TFTP server, and Telnet Prerequisites for ScaleIO Before starting a ScaleIO upgrade, ensure there is no rebalance or rebuild in progress on the ScaleIO system. If there is, wait for the rebalance or rebuild to finish before proceeding with the upgrade. Validate that the ScaleIO system is alert-free and that the underlying hardware has no failed components. Prerequisite tasks Complete the following tasks before you continue upgrading your system. Review the release notes and complete any required actions. 5 Upgrade prerequisites

6 Ensure that the appropriate security policies are applied to the Converged Systems. Upgrades do not modify Converged System security. Plan for the upgrade. Baseline security standards for Converged Systems are available in Converged Systems security documents. Calculate the upgrade time. Schedule adequate downtime (if required) for system maintenance. Ensure that all health checks have been completed. Confirm that all components are in a ready state with no alarms. Upgrade prerequisites 6

7 Upgrade best practices and planning This topic describes how to plan for an upgrade and the best practices for performing the upgrade. Use the following guidelines to ensure minimum impact on the production traffic: Review this entire upgrade document and each referenced component upgrade document before you begin the upgrade. Perform the upgrade during scheduled maintenance windows or non-disruptively when virtual machines are running. The best practice is to perform operational verification after each component, or logical grouping of components, has been upgraded. This should be taken into consideration during assessment and planning. Before upgrading, confirm the overall status and software version of each component. Verify that the component is in the ready state and is not generating any alarms or errors. When upgrading Converged System compute nodes or VxRack Controller servers, migrate virtual machines to another server in the VMware vsphere ESXi cluster before rebooting the Converged System compute nodes or VxRack Controller servers. The ScaleIO VMs (SVMs) cannot be migrated from their Converged System compute nodes. Create a configuration backup file for each Converged System component before you begin the upgrade. 7 Upgrade best practices and planning

8 Estimating the upgrade time This task allows you to estimate the amount of time you need to perform the upgrade. Plan your upgrade using the minimum time that it takes to upgrade the software on each component. The minimum time estimates do not include the time for preparation, configuration, VM migration, backup, and download. Review the list of components and determine the upgrade time for the components in the system. If a component is not in the system, do not estimate the upgrade time for that component. For cluster configurations, include the primary and the subordinate components. Order Component Upgrade time Notes 1. Cisco Nexus Switches (3000 series and 9000 series) 2. VxRack Controller components BMC, BIOS 3. Converged System compute nodes BIOS BMC Network Card StorCLI RAID Card 4. VMware vsphere Infrastructure Management (VIM) components VMware Platform Services Controller PSC-1 and PSC-2 VMware vcenter Server Virtual Appliance (vcsa) VMware vsphere Client VMware vsphere Update Manager Server VMware vsphere Update Manager Client Plug-in 2 hours 2 hours Migrate any VMs from the VxRack Controller Server that you are upgrading. Also, validate that the vsan is healthy after each reboot. 2 hours (per node) Upgrade one node at a time. Migrate any VMs before the upgrade and make sure the ScaleIO rebuild and rebalance is complete before upgrading the next node. 2-3 hours This is an estimate. 5. VMware Hypervisor ESXi 2-3 hours per host (allowing for ScaleIO rebuild/ rebalance) Upgrade one node at a time. Migrate any VMs before the upgrade and make sure the ScaleIO rebuild and rebalance is complete before upgrading the next node. 6. VSAN 2 hours When all nodes are upgraded, check that the VSAN is healthy before going to the next step. Estimating the upgrade time 8

9 Order Component Upgrade time Notes 7. ScaleIO components (upgrade from 1.32.x to ) ScaleIO plugin Gateway LIA Secondary MDM Primary MDM Tie breaker SDS Red Hat ScaleIO GUI SDC 2-3 hours Upgrade one node at a time. Migrate any VMs before the upgrade and make sure the ScaleIO rebuild and rebalance is complete before upgrading the next node. 9 Estimating the upgrade time

10 Upgrading Converged Systems releases This topic describes the upgrade sequence for system components. Upgrade the software and firmware on the Converged System in the order listed in the following table. Before upgrading, read the release notes and the release certification matrices for the applicable release on your Converged System. Some Converged System components are optional or specific to a customer environment. If the Converged System that you are upgrading does not have a component listed in the following table, skip that component and proceed with the next component. Order Component 1. Cisco Nexus 3000 and 9000 Series Switches Upgrade guide Cisco Nexus 3000 Series Switches: nexus-3000-series-switches/products-installation-guides-list.html Upgrade guide Cisco Nexus 9000 Series Switches: 2. VxRack Controller components: BMC and BIOS. See Upgrading BIOS on VxRack System compute nodes and Upgrading the VxRack System node BMC firmware for further information. 3. VxRack compute nodes (See Updating VxRack compute nodes for further information): BIOS BMC Network Card RAID Card StorCLI 4. VMware vsphere Infrastructure Management components VMware Platform Services Controller PSC-1 and PSC-2 VMware vcenter Server Virtual Appliance (vcsa) VMware vsphere Client VMware vsphere Update Manager Server VMware vsphere Update Manager Client Plug-in Software: Flexnet %2Fvcec%2Findex Use vsphere 6.0 documentation to upgrade: 5. VMware Hypervisor ESXi See the following upgrade issues for vsphere 6.0: release-notes.html (See Upgrading the VMware Hypervisor ESXi for further information) Upgrading Converged Systems releases 10

11 Order Component 6. vsan 7. ScaleIO components : ScaleIO plugin Gateway LIA Secondary MDM Primary MDM Tie breaker SDS Red Hat ScaleIO GUI SDC Upgrading VxRack Controller components About this task This topic describes how to update the BIOS and BMC firmware on VxRack Controller servers. The BMC firmware version installed on the nodes must match the RCM of the system. If it is not the correct version, you will need to update the firmware. Before you begin Download the current package as per the RCM from Flexnet. 1 Log on to the VxRack Controller jump server. 2 Download the relevant Extensible Firmware Interface (EFI) update package for your server board or system from the RCM repository. 3 Using Flexnet, copy all the files to a USB drive. The USB drive must be formatted as FAT16 or FAT32. 4 Plug in the USB device containing the system software updates into the host. 5 Migrate VMs that are running on the host, and put the host into maintenance mode. To enter maintenance mode, use the VMware vsphere Web Client, and put a host into maintenance mode. Then select Ensure Accessibility. 6 Power cycle the server. 7 During POST, press the F2 key when prompted to access the BIOS Setup Utility. 8 In BIOS Setup, move the cursor over to the BOOT MANAGER menu. 9 Move the cursor down to EFI and press Enter. The system boots to the embedded EFI shell. 11 Upgrading Converged Systems releases

12 10 Type the command map -r in the EFI shell to mount the USB Flash drive. 11 Type the command fs0: to change folders to the USB flash drive. The USB flash drive might also mount to fs1: depending on other installed devices. The number of fs# is dependent on the result of the map-r command. 12 Change folders to the folder storing the updated package files. 13 Type the command startup.nsh to start updating all the firmware and BIOS updates. 14 Reboot the system and enter the BIOS to verify the upgrade. 15 Connect to the BMC again and look for Firmware Version on the Dashboard to make sure that the BMC firmware is updated. 16 Exit maintenance mode on the controller host, and ensure vsan is healthy. From the VMware vsphere Web Client, navigate to the cluster, then to Monitor, then click vsan Health. Ensure that the vsan is not in reduced accessibility and wait for the rebuild to finish. 17 Migrate VMs back to the host. 18 Repeat Steps 4-17 for each controller host. Upgrading VxRack System compute nodes Upgrading BIOS on VxRack FLEX compute nodes Use this procedure to upgrade the BIOS on VxRack FLEX compute nodes. About this task This topic describes how to update the BIOS firmware on VxRack System compute nodes. The BIOS firmware version installed on the nodes must match the version listed on the RCM of the system. If it is not the correct version, you will need to update the BIOS firmware. Before you begin Before you start the upgrade, do the following: Obtain the IP address, username and password for the VxRack node BMCs. Copy the BIOS firmware package from the RCM repository to the VxRack Controller jump server. Shut down the SVM before you reboot the host during the upgrade. Perform this upgrade on one host at a time. Wait for the rebalance and rebuild to complete after each node upgrade (DO NOT proceed to the next host until all rebalances and rebuilds are complete, or if ScaleIO alerts are present). Observe MDM locations and ensure you do not power down the Master and Slave MDM at the same time, as this can cause a system outage. Upgrading Converged Systems releases 12

13 1 Log on to the VxRack Controller jump server. 2 Using a SSH client, connect to the ESXi host. 3 Migrate VMs that are running on the host, and put the host into maintenance mode. To enter maintenance mode, enter vim-cmd hostsvc/maintenance_mode_enter or vimsh -n -e hostsvc/maintenance_mode_enter 4 Launch a browser and open the VxRack node BMC interface using the BMC IP address. 5 From the Remote Control tab, select Server Power Control and power off the server. 6 From the Maintenance tab, select BIOS Update. 7 Click Enter Update Mode. 8 When prompted, attach the.bin image downloaded previously. 9 Click Update. 10 Power on the server after the update completes. 11 Launch the remote console and enter the BIOS to verify that the firmware was updated. 12 Exit maintenance mode by typing the following command: vim-cmd /hostsvc/ maintenance_mode_exit or vimsh -n -e /hostsvc/ maintenance_mode_exit 13 From the ScaleIO GUI, ensure the SDS is online and operational, and there are no rebuilds or rebalance operations occurring on any SDS before proceeding. 14 Repeat Steps 2-13 for each node. 15 Log out of the BMC and the jump server. Upgrading the VxRack System node BMC firmware Use this procedure to update the BMC firmware on VxRack System nodes. About this task This topic describes how to manually update the BMC firmware on VxRack System nodes. The BMC firmware version installed on the nodes must match the version listed on the RCM of the system. If it is not the correct version, you will need to update the firmware. Before you begin Before you start the upgrade, do the following: Obtain the IP address, username and password for the VxRack System node BMCs. Copy the BMC firmware package from the RCM repository, and copy it to the VxRack Controller jump server. 13 Upgrading Converged Systems releases

14 Shut down the SVM before you reboot the host during the upgrade. Perform this upgrade on one host at a time. Wait for the rebalance and rebuild to complete after each node upgrade (DO NOT proceed to the next host until all rebalances and rebuilds are complete, or if ScaleIO alerts are present). Observe MDM locations and ensure you do not power down the Master and Slave MDM at the same time as this can cause a system outage. 1 Log on to the VxRack Controller jump server. 2 Using an SSH client, connect to the ESXi host. 3 Migrate or power off VMs that are running on the host, and put the host in maintenance mode. To enter maintenance mode, enter vim-cmd hostsvc/maintenance_mode_enter or vimsh - n -e hostsvc/maintenance_mode_enter 4 Launch a browser and open the VxRack node BMC interface. 5 From the Maintenance tab, select BMC Firmware Update. 6 Select Preserve all configuration and click Enter Update Mode. A warning message appears. 7 Click OK. 8 When prompted, attach the.ima_enc image and click Upload. 9 Image verification takes place and the current and new image version displays on the screen. Verify the new version matches the RCM. 10 Click Proceed. A warning message appears. 11 Click OK. The firmware upgrade begins. 12 Connect to the BMC again and look for Firmware Version on the Dashboard to make sure that the BMC firmware is updated. 13 Exit maintenance mode by typing the following command: vim-cmd /hostsvc/ maintenance_mode_exit or vimsh -n -e /hostsvc/ maintenance_mode_exit. 14 From the ScaleIO GUI, ensure the SDS is online and operational, and there are no rebuilds or rebalance operations occurring on any SDS before proceeding. 15 Repeat Steps 2-14 for each node. 16 Log out of the BMC and the jump server. Upgrading the VxRack System node network card driver Use this procedure to update the network interface drivers according to the RCM on VxRack System nodes. Upgrading Converged Systems releases 14

15 Before you begin Copy the VIB package from Flexnet to the jump server. Verify that VMware Hypervisor ESXi is installed and configured. Verify that the SSH shell and ESX shell are enabled on the ESXi host. Verify the host network interface is reachable. Be sure that PuTTY and the SCP client are installed on the jump server. Ensure all VMs are migrated to another host before the upgrade. Shut down the SVMs (one SVM on one host at a time) before you reboot the host during the upgrade. Perform this upgrade on one host at a time. Wait for the rebalance and rebuild to complete after each node upgrade (DO NOT proceed to next host until all rebalances and rebuilds are complete, or if ScaleIO cluster alerts are present). Observe MDM locations and ensure you do not power down the Master and Slave MDM at the same time, as this can cause a system outage. 1 Log on to the VxRack Controller jump server. 2 Using an SSH client, connect to the ESXi host. 3 Migrate or power off VMs that are running on the host and put the host into maintenance mode. To enter maintenance mode, type: vim-cmd hostsvc/maintenance_mode_enter or vimsh -n -e hostsvc/maintenance_mode_enter 4 Use WinSCP to copy the VIB or offline bundle to the ESXi server /tmp folder. 5 Using SSH shell, connect to the ESXi host and check for the uploaded file by typing the following command: cd /tmp list 6 To install the driver, type: esxcli software vib install -d "/tmp/patchname.zip" 7 To update the driver, type: esxcli software vib update -d "/tmp/patchname.zip" 8 Verify that the VIB is installed from the shell: esxcli software vib list 9 Reboot the ESXi host. 10 Exit maintenance mode by typing: vim-cmd /hostsvc/maintenance_mode_exit or vimsh -n -e /hostsvc/maintenance_mode_exit 11 From the ScaleIO GUI, ensure the SDS is online and operational, and there are no rebuilds or rebalance operations occurring on any SDS before proceeding. 12 Repeat Steps 2-11 for each node. 15 Upgrading Converged Systems releases

16 13 Log out from the jump server. Upgrading StorCLI for ESXi nodes Use this procedure to upgrade StorCLI on ESXi nodes. Before you begin Complete this procedure on each ESXi host. Ensure all VMs are migrated to another host before the upgrade. 1 Open WinSCP and copy vmware-esx-storcli-1.1x.xx.vib to the host /tmp folder. 2 Use SSH to access the ESXi host and log on as root. 3 Type: esxcli software vib list grep storesxcli software vib remove n vmware-esx-storcli-1.1x.xx (output of the previous command)esxcli software vib install v /tmp/vmware-esx-storcli-1.1x.xx.vib -f 4 Verify that StorCLI is installed by typing /opt/lsi/storcli/storcli /c0 show. Upgrading StorCLI on ScaleIO VMs and Red Hat nodes Use this procedure to upgrade the StorCLI on ScaleIO VMs and Red Hat nodes. Before you begin Copy the updated StorCLI RPM from Flexnet to the jump server. Ensure that PuTTY and the WinSCP client are installed on the jump server. Verify that you can use SSH to access the ScaleIO VM. 1 Log on to the VxRack Controller jump server. 2 Using an SSH client, connect to the ScaleIO VM to be updated. 3 Use WinSCP to copy the StorCLI RPM to the ScaleIO VM server /tmp folder. 4 Type cd /tmp to navigate to the folder where the StorCLI RPM was copied. 5 Install the StorCLI RPM by typing the following command rpm -Uvh /tmp/ storcli-1.xx.xx-x.noarch.rpm. 6 Verify StorCLI version by typing the following command rpm -qa grep -i storcli. Upgrading Converged Systems releases 16

17 7 To upgrade StorCLI on Red Hat servers, use these steps for Red Hat servers instead of ScaleIO VMs to connect to the server to upload package and run the commands. Upgrading RAID card firmware and drivers on ScaleIO VMs Use this procedure to upgrade the RAID card according to the RCM on ScaleIO VMs. Before you begin Copy the updated StorCLI RPM from Flexnet to the jump server. Ensure that PuTTY and the WinSCP client are installed on the jump server. Verify that you can use SSH to access the ScaleIO VM. Ensure no rebuilds or rebalancing are occurring before proceeding with the upgrade from the ScaleIO GUI. Migrate all the VMs from the host (except ScaleIO VM) before the upgrade. Perform the update on one ScaleIO VM at a time. Wait for rebalance and rebuild to finish before proceeding. 1 Log on to the VxRack Controller jump server. 2 Log on to the ScaleIO GUI. Click the Backend tab, and right-click the SDS. Remove all the disks. 3 Wait for the rebuild and rebalance to complete before proceeding. 4 Using an SSH client, connect to the ScaleIO VM to be updated. 5 Use WinSCP to copy the driver and firmware to the ScaleIO VM server /tmp folder. 6 Using the SSH shell, connect to the ScaleIO VM. 7 Stop the SDS by typing /opt/emc/scaleio/sds/bin/delete_service.sh 8 Type: /opt/megaraid/storcli/storcli64 /c0 download file=/tmp/mr3108fw.rom mv /boot/grub/menu.lst /boot/grub/menu.lst.vce mv /boot/grub/menu.lst.bak /boot/grub/menu.lst vi /boot/grub/menu.lst Add elevator=noop to the kernel boot option. 9 Select ESC and type :wq! 17 Upgrading Converged Systems releases

18 10 Shut down the ScaleIO VM by typing shutdown -h now. 11 From the VMware vsphere Web Client: Ensure all VMs on the host where the ScaleIO VM is located are migrated off the host. Put the ESX host into maintenance mode. Reboot the ESX host. 12 After the ESX host is rebooted, take the host out of maintenance mode and power on the ScaleIO VM. 13 Using SSH, go back into the ScaleIO VM after the reboot is complete. 14 Navigate to the folder where the driver was copied using cd /tmp. 15 Install the drive RPM by typing: rpm -Uvh /tmp/lsi-megaraid_sas-kmp-default- XX.XXX.XX.XX_sles11sp3-2.x86_64.rpm 16 Unload the driver by typing: modprobe -r megaraid_sas. 17 Load the driver by typing: modprobe megaraid_sas. 18 Check the driver is current by typing: modinfo megaraid_sas grep version. version: srcversion: C87F38B7045FE2FEC0FEFC vermagic: default SMP mod_unload modversions 19 Using StorCLI check that the firmware and driver are the correct versions by typing: /opt/ MegaRAID/storcli/storcli64 /c0 show grep Version. Example: BIOS Version = _ _0x FW Version = Driver Version = Start the SDS by typing: /opt/emc/scaleio/sds/bin/create_service.sh. 21 Log out of the SSH session. 22 Click the ScaleIO GUI > Backend > Protection Domain (PD), and select the ScaleIO VM. Right-click and select Add Devices. 23 In the IP Address section type the IP Address of the host you removed and click OK. The host is re-added to the ScaleIO system. Upgrading Converged Systems releases 18

19 24 Use SSH to access the host running the ScaleIO VM from the jump server, and type fdisk -l 2>/dev/null grep /dev/sd grep -v sda. Note the device paths. Example: fdisk -l 2>/dev/null grep /dev/sd grep -v sda Disk /dev/sdc: GB, bytes Disk /dev/sdb: GB, bytes Disk /dev/sdd: GB, bytes Disk /dev/sde: GB, bytes Disk /dev/sdf: GB, bytes 25 From the ScaleIO GUI, click Backend and select the host. Right-click and select Add Devices. 26 Paste the device paths noted previously in each column. Ensure all the devices have been added. 27 Log out of the SSH session. 28 Wait for the rebuild or rebalance operation to finish, then proceed to the next node. 29 Log out of the jump server. Upgrading RAID card firmware and drivers on Red Hat nodes Use this procedure to upgrade the RAID card according to the RCM on Red Hat nodes. Before you begin Copy the updated StorCLI RPM from Flexnet to the jump server. Ensure that PuTTY and the WinSCP client are installed on the jump server. Verify that you can use SSH to access the nodes. Ensure no rebuilds or rebalancing are occurring before proceeding with the upgrade from the ScaleIO GUI. Migrate all the VMs from the host (except the ScaleIO VM) before the upgrade. Perform the update on one ScaleIO VM at a time. Wait for rebalance and rebuild to finish before proceeding. 1 Log on to the VxRack Controller jump server. 2 Log on to the ScaleIO GUI. Click on the Backend tab and right-click to select the SDS. Remove all disks. 3 Wait for the rebuild and rebalance to complete. 4 Using an SSH client, connect to the node to be updated. 19 Upgrading Converged Systems releases

20 5 Use WinSCP to copy the driver and firmware to the ScaleIO VM server /tmp folder. 6 Using the SSH shell, connect to the Red Hat node. 7 Stop the SDS by typing: /opt/emc/scaleio/sds/bin/delete_service.sh 8 Type: /opt/megaraid/storcli/storcli64 /c0 download file=/tmp/mr3108fw.rom 9 Use SSH to log on to the node after the reboot completes. 10 Navigate to the folder where the driver was copied using cd /tmp. 11 Install the drive RPM by typing: rpm -Uvh /tmp/kmodmegaraid_sas _el7.2-2.x86_64.rpm 12 Unload the driver by typing: modprobe -r megaraid_sas 13 Load the driver by typing: modprobe megaraid_sas. 14 Check the driver is current by typing: modinfo megaraid_sas grep version. version: srcversion: C87F38B7045FE2FEC0FEFC vermagic: default SMP mod_unload modversions 15 Using StorCLI, check that the firmware and driver are the correct versions by typing: /opt/ MegaRAID/storcli/storcli64 /c0 show grep Version Example: BIOS Version = _ _0x FW Version = Driver Version = Start the SDS by typing: /opt/emc/scaleio/sds/bin/create_service.sh. 17 On the ScaleIO GUI, click Backend > Protection Domain (PD), and select the ScaleIO VM. Right-click and select Add Devices. 18 In the IP Address section type the IP address of the host you removed and click OK. The host is re-added to the ScaleIO system. Upgrading Converged Systems releases 20

21 19 Use SSH to access the host running the ScaleIo VM from the jump server, and type lsblk. Note the device paths. Example: 20 From the ScaleIO GUI, click Backend and select the host. Right-click and select Add Devices. 21 Paste the device paths noted previously in each column. Ensure all the devices have been added. 22 Log out of the SSH session. 23 Wait for the rebuild or rebalance operation to finish, then proceed to the next node. 24 Log out of the jump server. Upgrading VMware vsphere infrastructure management components Use this procedure to upgrade VMware vsphere infrastructure management components. 21 Upgrading Converged Systems releases

22 About this task The following management components will be upgraded: VMware Platform Services Controller (PSC-1) and (PSC-2) VMware vcenter Server Virtual Appliance (vcsa) VMware vsphere Update Manager Server Before you begin Back up the VMware vsphere infrastructure management components. Download the PSC patch FP-ISO file from the RCM repository to the jump server in the VxRack Controller cluster. Take a snapshot of the vcsa and PSCs prior to upgrading. Upgrading the primary PSC, secondary PSC, and vcsa These steps are for upgrading major versions of the PSCs and vcsa. 1 Upload the VCSA Patch-FP ISO and the VUM VCSA-ALL ISOs to the vsan datastore on the VxRack Controller cluster via the VMware vsphere Web Client. 2 From the VMware Web Client, click Storage > vsan Datastore > Manage > Files. a b c Create a new folder named ISO (if not created already). Select the upload icon and upload the required ISO files. Allow time for the upload to complete. You can view the status at the bottom of the screen. 3 Mount the Patch-FP ISO to the VCSA, PSC1, and PSC2 appliances. a b c d e From the VMware Web Client, select the VM to attach the ISO. On the Summary screen, expand VM Hardware. On CD/DVD Drive, select the ISO. Browse to the vsan datastore, click the ISO folder, and select the correct ISO. Click OK. Note the IP addresses of each VM. These will be used later. 4 Open Mozilla Firefox on the jump server and navigate to the IP address of PSC2 on port 5480 as noted in the previous step. For example: 5 Log on to the interface with username root and default password VMwar3!!. 6 On the left menu, select Update > Check Updates > CDROM. Wait while the system validates the ISO attached earlier. Upgrading Converged Systems releases 22

23 7 When complete, select Install Updates. 8 After the upgrade completes, the system prompts you to reboot. Click Summary > Reboot. Allow up to 10 minutes for the VM to reboot. 9 Log back on to the VMware vsphere Web Client and validate that the SSO domain is running. 10 Upgrade PSC1 using the same procedure. 11 When both PSC1 and PSC2 are upgraded, launch the vcsa management user interface. 12 Log on to the interface with username root and default password VMwar3!!. 13 On the left menu, select Update > Check Updates > CDROM. Wait while the system validates the ISO attached earlier. 14 When validation is complete, select Install Updates. 15 After the upgrade is complete, the system prompts you to reboot. Click Summary > Reboot. Allow up to 10 minutes for the VM to reboot. 16 Log on to the VMware Web Client again. This may take some time to start, as the VCSA appliance can take up to 15 additional minutes to start all the vcenter services. Upgrading VMware vsphere Update Manager Server 1 Mount the VMware-VIMSetup-all ISO to the VUM appliance. a b c d From the VMware Web Client, select the VM to attach the ISO. On the Summary screen, expand VM Hardware. On CD/DVD drive, select the ISO. Note the IP address of each VM. You will need these later. 2 Use RDP to access the VUM server IP address, as administrator, with default password of VMwar3!! and launch the installer from My Computer. Browse to the CD/DVD drive, click autorun.exe, then click Yes. 3 Select vsphere Update Manager > Server and click Install. 4 Click Yes at the prompt to continue the upgrade. 5 Click OK. 6 Click Next. 7 Accept the EULA and click Next. 8 Clear the Download updates from default sources immediately after installation checkbox. 9 Click Next. 23 Upgrading Converged Systems releases

24 10 Set the following on the vcenter Server Information page: IP Address/Name: the VMware vcenter Server Fully Qualified Domain Name (FQDN). HTTP Port: 80 Password: the password associated with 11 Click Next. 12 Select Database Information and click Next. 13 Select Database re-initialization warning > Do not overwrite, leave my existing database in place and click Next. 14 Click Install, OK, Finish, and Exit. 15 Unmount the VIM ISO. 16 Validate the functionality of all components. 17 Delete the snapshots of the PSC's and the vcsa that were taken prior to starting the upgrade activities. Upgrading the VMware vsphere Hypervisor ESXi using the VMware Update Manager About this task Use this procedure to upgrade VMware vsphere Hypervisor ESXi for VxRack Controller servers, and VxRack System nodes with ScaleIO VMs, using the VMware Update Manager (VUM). Before you begin Obtain login details for the customer VMware vcenter and ESXi hosts. Download the patch zip file from the RCM repository to the jump server in the VxRack Controller Cluster. Confirm that the jump server has PuTTY and WinSCP installed. Install if necessary. Disable vsphere HA on the cluster for remediation using the VUM. Due to ScaleIO VMs, remediate VxRack System nodes one at a time. Wait for the rebalance and rebuild to complete after each node upgrade (DO NOT proceed to next host until all rebalances and rebuilds are complete, or if ScaleIO cluster alerts are present). Observe MDM locations and ensure you do not power down the Master and Slave MDM at the same time, as this can cause a system outage. Upgrading Converged Systems releases 24

25 Upgrade process for the VxRack ESXi hosts. 1 Log on to the jump server and launch a web browser and connect to the VMware vsphere Web Client. 2 Log on to the VxRack VMware vsphere Web client. 3 From the VMware vsphere Web client Home screen, click Update Manager under Monitoring. 4 Select Update Manager Server. 5 Select Manage and click Patch Repository. 6 Upload the ESXi patches by selecting Import Patches to launch the Import Patches pop-up. 7 Click Browse to browse to the location where you saved the patch archive (.zip) and then click Upload file and wait for the patch upload to complete. Click Next and Finish. You do not need to extract the contents of the.zip archive. The Update Manager understands the format of the.zip archive and extracts the contents as it imports the patches. If the import fails, verify the checksum of the.zip archive to make sure the file did not get corrupted during the download. 8 With the patches loaded into Update Manager, create a baseline group that you can use to remediate your hosts. Select To create a Baseline from the Update Manager home screen, select the VUM > Hosts Baseline > Add Baseline. The New Baseline wizard with steps for creating a new baseline starts. 9 Name the baseline and select Host Patch as the baseline type. It is recommended that the baseline name coincide with the patch name used by VMware for easier tracking. Click Next. 10 Select either the Fixed or Dynamic patch option. Click Next. 11 Select the patches for the baseline. Click Next. 12 Review the baseline information, ensuring the required patches are listed. Click Finish. 13 Return to the Host Baselines tab and verify the patches appear in the list. 14 Click Go to compliance view. 15 Click Attach Baseline. From the pop-up window, select the baseline and click OK. 16 Go to Home > Hosts and Clusters > Select Cluster > Manage > vsphere HA and click Edit. Uncheck Host Monitoring and turn on vsphere HA. Click OK. 17 Go to Home > Hosts and Clusters. Select the host to remediate. If upgrading a Controller server, right-click Host > Maintenance Mode > Enter Maintenance Mode and select Ensure Accessibility and click OK. If upgrading a Compute server, right-click the associated ScaleIO VM > Power > Shut Down Guest OS and select Yes. Right-click Host > Maintenance Mode > Enter Maintenance Mode and select OK. 18 Go to the Manage tab and click Update Manager. Verify that the baseline is attached. Click Scan for Updates. Ensure that Baseline is checked and click Remediate. Click Next. Ensure that the host is checked and click Next. 25 Upgrading Converged Systems releases

26 19 Ensure all the patches are checked and click Next. 20 In the Schedule window, optionally change Task Name and/or add a Task description. Select Run this action now and click Next. 21 On the Host remediation options screen, accept all of the defaults and click Next. 22 On the Cluster remediation options screen, accept all of the defaults and click Next. 23 Review the settings on the Ready to complete screen. When the settings are verified, click Finish to start remediation. 24 Go to the Home screen. For Controller severs, select Hosts and Clusters. Right-click Host and select Maintenance Mode > Exit Maintenance Mode. Ensure that the vsan is healthy before moving to the next node. For Compute servers, select Hosts and Clusters. Right-click Host and select Maintenance Mode > Exit Maintenance Mode. Right-click the associated ScaleIO VM > Power > Power On. 25 From the ScaleIO GUI, ensure the SDS is online and operational, and there are no rebuilds or rebalance operations occurring on any SDS before proceeding. 26 Repeat Steps for all clusters and hosts. Verify all hosts and systems are up and running. When patching is complete, log out of the program. 27 Go to the Home screen and select Hosts and Clusters. Select the cluster, click Manage, select vsphere HA and click Edit. Check Host Monitoring, enable vsphere HA, and click OK. Update Manager works best if DRS is enabled in fully automated mode, allowing the VMs to be migrated off the host as part of the remediation. If DRS is not running in fully automated mode, you need to manually migrate or shutdown the VMs prior to the remediation. Upgrading VMware vsphere ESXi using the ZIP patch and network driver on VxRack Controller hosts Use this procedure to upgrade the VMware ESXi for VxRack Controller servers using the ZIP patch and network driver. Before you begin Obtain login details for the Controller VMware vcenter and ESXi hosts. Download the ZIP files from the RCM repository to the jump server in the VxRack Controller cluster. Confirm that the jump server has PuTTY and WinSCP installed. Remediate the VxRack Controller nodes one at a time. 1 Log on to the jump server, launch a web browser, and connect to the VMware vsphere Web Client. Upgrading Converged Systems releases 26

27 2 Log on to the VxRack VMware vsphere Web Client. 3 From the Web Client, select Cluster > Health > Virtual SAN > Health. Ensure that the VSAN is healthy before moving to the next node. 4 From the Home screen, select Hosts and Clusters. 5 Right-click Host and select Maintenance Mode > Enter Maintenance Mode. 6 Ensure all VMs migrate off the host. If not, manually migrate them by right clicking and selecting Migrate. 7 Use WinSCP to copy the ESXi Build zip patch file and the netixgbe_ oem zip network driver to the VSAN datastore ISO folder on the ESXi server. 8 Using the SSH shell, connect to the ESXi host and check for the uploaded file by typing the following command: cd /vmfs/volumes/vsandatastore/iso. 9 To install the driver, type: esxcli software vib install -d /vmfs/volumes/ vsandatastore/iso/net-ixgbe_ oem zip. 10 To install the patch, type: esxcli software vib install -d /vmfs/volumes/ vsandatastore/iso/esxi build zip. 11 Reboot the ESXi host. 12 From the Home screen, select Hosts and Clusters. 13 Right-click Host and select Maintenance Mode > Exit Maintenance Mode. 14 From the VMware vsphere Web Client, select Cluster > Health > Virtual VSAN > Health. Ensure that the vsan is healthy before moving to the next node. 15 Repeat steps for all VxRack Controller hosts. Upgrading the VMware vsphere Hypervisor ESXi using the ZIP patch file and network driver on ScaleIO nodes Use this procedure to upgrade the VMware vsphere Hypervisor ESXi using the ZIP patch file and network driver on ScaleIO nodes. Before you begin Obtain login details for the customer VMware vcenter and ESXi hosts. Download the ZIP files from the RCM repository to the jump server in the VxRack customer cluster. Confirm that the jump server has PuTTY and WinSCP installed. Remediate one server at a time. 27 Upgrading Converged Systems releases

28 Wait for the rebalance and rebuild to complete after each node upgrade. Do not proceed to the next host until all rebalances and rebuilds are complete and no ScaleIO cluster alerts are displayed. Take special care to observe MDM locations and ensure you do not power down the primary and secondary MDM at the same time as this can cause a system outage (this should not be an issue as long as one node is done at a time). 1 Log on to the jump server and launch a web browser to connect to the customer VMware vsphere Web Client. 2 Open the ScaleIO GUI. 3 Select the SDS running on the ESX host you wish to update, and put it into maintenance mode. 4 Log on to the VMware vsphere Web Client. 5 From the Home screen, select Hosts and Clusters. 6 Right-click Host and select Maintenance Mode > Enter Maintenance Mode. 7 Ensure all VMs migrate off the host. If not, manually migrate them by right clicking the VMs and select Migrate. 8 Power off the ScaleIO Virtual Machine (SVM). SVMs are stored on local storage and cannot be vmotioned. 9 Use WinSCP to copy the ESXi Build zip patch file and the netixgbe_ oem zip network driver to the DASXX folder on the ESXi server (where XX is unique for each host). 10 Using the SSH shell, connect to the ESXi host and check for the uploaded file by typing the following command: cd /vmfs/volumes/dasxx/ 11 To install the driver, type: esxcli software vib install -d /vmfs/volumes/dasxx/ net-ixgbe_ oem zip 12 To install the patch, type: esxcli software vib install -d /vmfs/volumes/dasxx/ ESXi Build zip 13 Reboot the ESXi host. 14 From the VMware vsphere Web Client Home screen, select Hosts and Clusters. 15 Right-click Host and select Maintenance Mode > Exit Maintenance Mode for ScaleIO nodes. 16 Power on the SVM on the host. 17 Open the ScaleIO GUI. 18 Ensure the SDS is online and operational and there are no rebuilds are rebalance operations occurring on any SDS before proceeding. Upgrading Converged Systems releases 28

29 19 Repeat these steps for each ScaleIO node. Upgrading ScaleIO About this task Use this procedure to upgrade an existing ScaleIO system. Before you begin Before starting the upgrade, you need to: Log on to the ScaleIO GUI and verify that the system is error-free and is not currently in rebuild or rebalance states. Do not attempt to upgrade the system if there are errors or a rebuild/rebalance in progress. Resolve all issues and wait until the rebuild/rebalance is complete before you begin the upgrade procedure. Start the Scale IO GUI and verify the following: No rebuild or rebalance is running in the background No degraded capacity exists No SDS or SDC is disconnected No SDS device is in an error state The MDM cluster is not in degraded mode, including the tie-breaker Obtain the username and passwords to all the components of the ScaleIO System, including SVM, MDM, Customer vcenter, and ESXi. Confirm that PuTTY and WinSCP are installed on the jump server, and install them if they are not. Download the following packages to the jump server: ScaleIO_ 1.32.X_Complete_VMware_SW_Download.zip ScaleIO_1.32.X_RHEL7_Download.zip ScaleIO_1.32.X_Gateway_for_Linux_Download.zip ScaleIO_1.32.X_Complete_Windows_SW_Download.zip When upgrading ScaleIO, upgrade components in the following sequence: ScaleIO GUI ScaleIO VMware vsphere web plugin Deleting and uploading the OVA template Gateway LIA 29 Upgrading Converged Systems releases

30 Secondary MDM Primary MDM Tie-breaker SDS RedHat SDC Upgrading the ScaleIO GUI Use this procedure to upgrade the ScaleIO GUI. To upgrade the ScaleIO GUI, on the jump server, download and type EMC-ScaleIO-gui XXX.X.msi. Removing the ScaleIO plug-in from VMware vcenter Use the following steps to remove the ScaleIO plug-in from VMware vcenter. 1 Using the console or remote desktop, access the VxRack Controller jump server. 2 Create a directory: D:\SIOUP132X\InstallFiles. 3 Extract the ScaleIO_1.32.X_Complete_VMware_SW_Download.zip to D:\SIOUP132X \InstallFiles. 4 Create a directory: D:\SIOUP132X\Plugin. 5 Unzip the D:\SIOUP132X\InstallFiles \ScaleIO_1.32.X_Complete_VMware_SW_Download \ScaleIO_1.32.X_vSphere_Plugin_Directory\EMC-ScaleIO-vSphere-webplugin-package XXXX.X.zip into D:\SIOUP132X\Plugin. 6 Run the script to remove the plug-in: a b From the D:\SIOUP132X\Plugin directory use PowerCLI to run the ScaleIOPluginSetup-1.32-XXXX.X.ps1 script. Select option 2, Unregister ScaleIO plugin. 7 Enter the VMware vcenter credentials, (using the username domain\username format) and confirm the script actions. 8 Select option 2, Unregister ScaleIO plugin. Upgrading Converged Systems releases 30

31 9 Confirm the script result: Welcome to ScaleIO Plugin Unregister Utility Connecting to vcenter IP... Successfully connected to vcenter IP. ScaleIO vsphere client plugin successfully removed. Log out and log in to vsphere web client to complete the unregistration process 10 Log out, then log back in to the VMware vsphere Web Client. The plug-in is no longer registered. Registering the new ScaleIO plug-in Use the following steps to register the new ScaleIO plug-in. Before you begin, ensure that there is communication between the VMware vsphere web client server (usually installed on the vcenter) and the web server storing the plug-in. 1 Using PowerCLI for VMware, as an administrator, run Set-ExecutionPolicy AllSigned. Enter Y. 2 From PowerCLI, go to the D:\SIOUP132X\Plugin directory. 3 Run the following script: ScaleIOPluginSetup-1.32-XXXX.X.ps1. Enter R. a b c d e Enter the vcenter name or IP address, user name (using the domain/username format rather than username@domain), and password. For Choose Mode, choose option 1, Register SIO plugin. Read the upgrade notice, and enter Y to continue. For Select Registration Mode, choose Standard. This step may take a few minutes. If necessary, accept the thumbprint. 4 Log on to the VMware vsphere Web Client. If you are already logged on, log off, and then log on again. 5 In the PowerCLI window, press Enter to finish the plug-in download and return to the menu. 6 From the vsphere Home tab, verify that the ScaleIO icon is visible in the Inventories section. 7 If the icon is not visible, connect to the vcenter Server Appliance as root via SSH. Run the following command in the SSH client: shell.set --enabled true > shell > service vsphere-client restart If the ScaleIO System is not visible in VMware vcenter, re-register the system by clicking the ScaleIO icon and clicking Register System. Enter the MDM IP address, username, password, and click OK. 31 Upgrading Converged Systems releases

32 Deleting and uploading the OVA template Use the following steps to delete the old OVA template and upload the new OVA template. 1 Log on to the VMware vsphere web client. 2 Select VMs and Templates. 3 Expand the folder that begins with ScaleIO 4 Right-click on the each template named EMC ScaleIO SVM Template (v1.32.xxxx.x) and select Delete from Disk. 5 Go back to the Windows jump server to create a directory: D:\SIOUP132X\OVA. 6 Copy D:\SIOUP132X\InstallFiles \ScaleIO_1.32.X_Complete_VMware_SW_Download\ScaleIO_1.32.X_ESX_Download \ScaleIOVM_1.32.XXXX.X.ova to D:\SIOUP132X\OVA. 7 Open PowerCLI as administrator. 8 Run ScaleIOPluginSetup-1.32-xxx.x.ps1 from the D:\SIOUP132X\Plugin folder. 9 Enter the vcenter name or IP address, user name (using the domain/username format rather than username@domain), and password. 10 Select option 3 and enter the datacenter name. 11 Enter the path to OVA including the OVA file name. For example: D:\SIOUP132X\OVA \ScaleIOVM_1.32.XXXX.X.ova. 12 Enter the VxRack vcenter datastore from which the template will deploy. On deployments of over 100 nodes, use up to 3 datastores to speed up the deployment process. 13 Enter a blank datastore name to indicate to the script that no further datastores are to be used and to continue to upload the OVA file. The upload process starts. 14 Enter 4 to exit the script. Upgrading the Gateway Use the following steps to upgrade the gateway. 1 From the jump server SCP, copy ScaleIO_1.32.X_Gateway_for_Linux_Download.zip to the Gateway SVM into /tmp. 2 Use SSH to access the Gateway SVM. Upgrading Converged Systems releases 32

33 3 Extract the gateway RPM by typing: a b cd /tmp unzip ScaleIO_1.32.X_Gateway_for_Linux_Download.zip 4 Type the following command: SIO_GW_KEYTOOL=<keytool_path> rpm -Uvh /tmp/emc- ScaleIO- gateway-1.32-xxxx.x.noarch.rpm. For example: SIO_GW_KEYTOOL=/usr/ java/default/bin/ rpm -Uvh /tmp/emc-scaleio-gateway XXXX.X.noarch.rpm 5 Verify the new version: rpm -qa grep EMC-ScaleIO-gateway. For example ScaleIO GW:~ # rpm -qa grep EMC-ScaleIO-gateway EMC-ScaleIO-gateway Upgrading LIA Use the following steps to upgrade LIA. 1 From the jump server use SCP to copy the following file into the /tmp folder of all SVMs: D: \SIOUP132X\InstallFiles\ScaleIO_1.32.X_Complete_VMware_SW_Download \ScaleIO_1.32.X_SLES_11.3_SVM_Download\*.*. 2 Use SSH to access all SVMs and type mkdir /tmp/scaleio. 3 Type the following command on all SVMs: rpm Uvh /tmp/emc-scaleio-lia XXXX.X.sles11.3.SVM.x86_64.rpm. 4 Verify the new version by typing: rpm -qa grep EMC-ScaleIO-lia. For example: ScaleIO :~ # rpm -qa grep EMC-ScaleIO-lia EMC-ScaleIO-lia sle11.3 Upgrading the Secondary MDM Use the following steps to upgrade the Secondary MDM. 1 Use SSH to access the Secondary MDM as root and type the following command: rpm Uvh /tmp/emc-scaleio-mdm-1.32-xxxx.x.sles11.3.svm.x86_64.rpm. 2 SSH to the primary MDM, log on asroot and admin and type: scli --login --username admin --password <admin password> run scli --switch_mdm_ownership. 33 Upgrading Converged Systems releases

34 3 Verify the new version by typing: rpm -qa grep EMC-ScaleIO-mdm. For example: ScaleIO :~ # rpm -qa grep EMC-ScaleIO-mdm EMC-ScaleIO-mdm sle11.3 Upgrading the Primary MDM Use the following steps to upgrade the primary MDM. 1 Use SSH to access the other MDM as root and type the following command: rpm Uvh /tmp/ EMC-ScaleIO-mdm-1.32-XXXX.X.sles11.3.SVM.x86_64.rpm. 2 Verify the new version: rpm -qa grep EMC-ScaleIO-mdm. For example: ScaleIO :~ # rpm -qa grep EMC-ScaleIO-mdm EMC-ScaleIO-mdm sle11.3 Upgrading the Tie-breaker Use the following steps to upgrade the Tie-breaker. 1 Use SSH to access the Tie-breaker as root and type the following command: rpm Uvh /tmp/ EMC-ScaleIO-tb-1.32-XXXX.X.sles11.3.SVM.x86_64.rpm. 2 Verify the new version: rpm -qa grep EMC-ScaleIO-tb. For example: ScaleIO :~ # rpm -qa grep EMC-ScaleIO-tb EMC-ScaleIO-tb sle11.3 Upgrading the SDS Use the following steps to upgrade the SDS. 1 Use SSH to access all SVMs (except the gateway) as root and type the following command: rpm Uvh /tmp/emc-scaleio-sds-1.32-xxxx.x.sles11.3.svm.x86_64.rpm. 2 Verify the new version: rpm -qa grep EMC-ScaleIO-sds. For example ScaleIO :~ # rpm -qa grep EMC-ScaleIO-sds EMC-ScaleIO-sds sle11.3 Upgrading Converged Systems releases 34