Nexus 9500 Spine Supervisor Failure Recovery

Transcription

1 Nexus 9500 Spine Supervisor Failure Recovery Contents Introduction Problem Solution Prepare to Convert the Replacement Supervisor to ACI Mode Procedure 1 - Convert from NX-OS to ACI Procedure 2 - Convert from Loader to ACI Introduction This document describes the process to replace a Spine's supervisor, which has failed due to a hardware error, in Application Centric Infrastructure (ACI) mode. Problem The active supervisor has failed due to a hardware failure. The standby supervisor has taken over as the active, reloaded into a fabric discovery state, and automatically joined the fabric. A replacement supervisor has been received and needs to be installed and converted to ACI mode. Solution Note: Conversion of the supervisor to ACI mode requires downtime for the spine as some components need to be disconnected. Prepare to Convert the Replacement Supervisor to ACI Mode Complete these steps in order to convert the replacement supervisor: 1. Remove the failed supervisor from the spine chassis. 2. Disconnect each ACI line card and the working active supervisor. It is not required for the line cards and active supervisor to be completely removed from the chassis. Then, insert the replacement supervisor. If the replacement supervisor boots into standalone mode (NX-OS) it needs to be converted to ACI mode by Procedure 1. Otherwise, if it boots into a loader> prompt follow Procedure 2.

2 Procedure 1 - Convert from NX-OS to ACI 1. Complete one of these three options in order to load an ACI image into the replacement supervisor's bootflash: Option A - Copy the ACI image from a USB drive into bootflash. List the contents of the USB drive (usb1: or usb2:) for the ACI image name in order to complete this:switch# dir usb1: switch# copy usb1:aci-n9000-dk f.bin bootflash:option B - Enable Secure Copy (SCP) services in order to push the ACI image from an Application Policy Infrastructure Controller (APIC). Configure the management 0 interface with an IP address, set up a default gateway for the management virtual routing and forwarding (VRF) instance, and enable the SCP server feature. switch# configure terminal switch(config)# interface mgmt 0 switch(config-if)# ip address ipv4-address{ [/length] [subnet-mask]} switch(config-if)# no shutdown switch(config-if)# exit switch(config)# vrf context management switch(config-vrf)# ip route /0 default-gw-ip switch(config-vrf)# exit switch(config)# feature scp-server switch(config)# exit switch# copy running-config startup-config Note: Before you proceed, verify the APIC can be pinged from the management VRF.Enter this command on the APIC:admin@apic:~> scp /firmware/fwrepos/fwrepo/<aci_image.bin> admin@<node-mgmt-ip>:<aci_image.bin> Note:

3 <aci_image.bin> is the filename of the ACI switch image located on the APIC. <nodemgmt-ip> is the management IP address which was previously configured on the switch.tip: ACI switch image names always begin with 'aci-n9000'.option C - Copy the image from an accessible SCP/FTP/TFTP server. Configure the management 0 interface with an IP address, set up a default gateway for the management VRF instance, and copy the image into bootflash. This example uses SCP. switch# configure terminal switch(config)# interface mgmt 0 switch(config-if)# ip address ipv4-address{ [/length] [subnet-mask] } switch(config-if)# no shutdown switch(config-if)# exit switch(config)# vrf context management switch(config-vrf)# ip route /0 default-gw-ip switch(config-vrf)# end Verify the server can be pinged from the management VRF and then copy the image from the server into bootflash.switch# copy scp://scpuser@ /path/to/aci-n9000-dk f.bin bootflash: vrf management Verify the ACI image has transferred successfully to the standalone switch with the dir bootflash: command. 2. Convert the supervisor to ACI mode. If the supervisor runs NX-OS version 6.1(2)I3(3) or above, use Method 1. Otherwise, use Method 2. Method 2 might also be used as a backup if Method 1 fails. Method 1 NXOS software tested: n9000-dk i3.3a.bin ACI software tested: aci-n9000-dk f.bin On the standalone node enter these commands:switch# configure terminal switch(config)# boot aci bootflash:<aci_image.bin> switch(config)# reloadmethod 2 NXOS software tested: n9000-dk i2.1.bin ACI software tested: aci-n9000-dk j.1.0-2j.bin On the standalone node enter these commands:switch# configure terminal switch(config)# no boot nxos switch(config)# end switch# copy running-config startup-config switch# reload This causes the switch to boot into the 'loader>' prompt. Enter the dir command in order to list the contents of bootflash for the ACI image name.

4 Boot the ACI image with the boot<aci_image_name.bin> command.the supervisor boots into ACI mode in a fabric discovery state. Log in with the username admin with no password. 3. Set the boot variables with the image name from the dir command. (none)# dir bootflash (none)# setup-bootvars.sh <aci_image.bin> 4. Enter these commands in order to verify the boot variables were set properly: leaf# cat /mnt/cfg/0/boot/grub/menu.lst.local boot aci-n9000-dk j.bin leaf# cat /mnt/cfg/1/boot/grub/menu.lst.local boot aci-n9000-dk j.bin 5. Reconnect all previously disconnected line cards as well as the other supervisor. The spine rejoins the fabric within a few minutes and the APIC pushes down all relevant policies to the new supervisor. Procedure 2 - Convert from Loader to ACI 1. Boot an ACI image with one of these two options. Option A - Boot from a USB drive. List the contents of the USB drive (usb1: or usb2:) for the ACI image name, then boot it from the USB. loader> dir usb1: loader> boot usb1:aci-n9000-dk f.bin Option B - Boot from an accessible TFTP server. Configure an IP address and a default gateway and then boot from the server. loader> set ip <ip_addr> <mask> loader> set gw <ip_addr>

5 loader> boot tftp://<tftp_path> An example is shown here:loader> set ip loader> set gw loader> boot tftp:// /aci-n9000-dk f.bin The supervisor boots into ACI mode in a fabric discovery state. Log in with the username admin with no password. 2. Set the boot variables on the supervisor with the image name from the dir command. (none)# dir bootflash (none)# setup-bootvars.sh <aci_image_name.bin> 3. Reconnect all previously disconnected line cards as well as the other supervisor. The spine rejoins the fabric within a few minutes and the APIC pushes down all relevant policies to the new supervisor. You will be prompted to log in with the fabric credentials. *Note: Faults F1582 "FPGA version mismatch detected" may be reported upon following the procedure. These faults will be resolved the first time you perform upgrade using firmware/maintenance policy from APIC.