Mellanox Firmware Tools (MFT) Release Notes. Rev 4.6.0

Transcription

1 Mellanox Firmware Tools (MFT) Release Notes Rev Mellanox Technologies

2 NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT ( PRODUCT(S) ) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS-IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale, CA U.S.A. Tel: (408) Fax: (408) Copyright Mellanox Technologies Ltd. Rights Reserved. Mellanox, Mellanox logo, Accelio, BridgeX, CloudX logo, CompustorX, Connect-IB, ConnectX, CoolBox, CORE-Direct, EZchip, EZchip logo, EZappliance, EZdesign, EZdriver, EZsystem, GPUDirect, InfiniHost, InfiniBridge, InfiniScale, Kotura, Kotura logo, Mellanox CloudRack, Mellanox CloudXMellanox, Mellanox Federal Systems, Mellanox HostDirect, Mellanox Multi-Host, Mellanox Open Ethernet, Mellanox OpenCloud, Mellanox OpenCloud Logo, Mellanox PeerDirect, Mellanox ScalableHPC, Mellanox StorageX, Mellanox TuneX, Mellanox Connect Accelerate Outperform logo, Mellanox Virtual Modular Switch, MetroDX, MetroX, MLNX-OS, NP-1c, NP-2, NP-3, Open Ethernet logo, PhyX, PlatformX, PSIPHY, SiPhy, StoreX, SwitchX, Tilera, Tilera logo, TestX, TuneX, The Generation of Open Ethernet logo, UFM, Unbreakable Link, Virtual Protocol Interconnect, Voltaire and Voltaire logo are registered trademarks of Mellanox Technologies, Ltd. other trademarks are property of their respective owners. For the most updated list of Mellanox trademarks, visit Document Number: Mellanox Technologies 2

3 Table of Contents Table of Contents List of Tables Chapter 1 Overview Package Tools Software Dependencies Supported Operating Systems and Platforms Supported Flash Types Supported Mellanox ICs Chapter 2 Changes and New Features in Rev Chapter 3 Known Issues Chapter 4 Bug Fixes History Chapter 5 History of Changes and New Features Rev Mellanox Technologies 3

4 List of Tables Table 1: Release Update History Table 2: Mellanox Firmware Tools (MFT) Available Tools Table 3: MFT Software Dependencies Table 4: Supported Operating Systems and Platforms Table 5: Supported Flash Types Table 6: Mellanox IC Devices Table 7: Changes and New Features in Rev Table 8: Known Issues and Limitations Table 9: Bug Fixes History Table 10: History of Changes and New Features Mellanox Technologies Rev 4.6.0

5 Release Update History Table 1 - Release Update History Release Date Description Rev February 2, 2017 Initial release of this MFT version Rev Mellanox Technologies 5

6 1 Overview These are the release notes for Rev of the Mellanox Firmware Tools (MFT). This release supports the following operating systems:, Windows, VMware ESXi and FreeBSD. Please see the supported platform table for further details. The tools functionality is identical in all operating systems unless otherwise noted. 1.1 Package Tools The following is a list of the available tools in the package, together with a brief description of each tool. The tools apply to single switch systems or adapter cards. The MFT tools do not provide cluster wide functionality. Table 2 - Mellanox Firmware Tools (MFT) Available Tools (Sheet 1 of 3) Category Tool Description MST Service mst Lists the available mst devices Start/stop the register access driver for and VMware ESXi OSs. Firmware Update and Configuration mlxburn flint mlxconfig This tool provides the following functions: Generating a standard or customized Mellanox firmware image for burning in.bin format Burning an image to the Flash attached to a Mellanox HCA or switch device Querying the firmware version loaded on a Mellanox device. Displaying the Vital Product Data (VPD) of a Mellanox network adapter This tool burns a firmware binary image or an expansion ROM image to the Flash of a Mellanox network adapter/switch device. It includes query functions to the burnt firmware image and to the binary image file. ows the user to change some of the device configurations without having to create and burn a new firmware. Operating System 6 Mellanox Technologies Rev 4.6.0

7 Overview Table 2 - Mellanox Firmware Tools (MFT) Available Tools (Sheet 2 of 3) Category Tool Description Firmware Update and Configuration Debug and Diagnostics Utilities mlxfwmanager The mlxfwmanager is a Mellanox firmware update and query utility. It provides a simple 'single click' firmware update functionality. Note: The same tool with embedded firmware binaries is released separately and is named mlxup. mlxphyburn A tool for burning externally managed PHY mlx_fpga A tool for burning and debugging Mellanox devices with FPGA. It allows the user to burn their own hardware code on an FPGA integrated with Mellanox HCA board. It also provides the user with read/write registers in the QDR memory of the FPGA. cpld_update itrace fwtrace mlxtrace mlxdump mlxmcg wqdump i2c mlxi2c mget_temp pckt_drop A tool for programing on board CPLDs for Mellanox devices for the OEM packages only. Extracts and prints trace messages generated by the firmware of a ConnectX-3 adapter cards. Extracts and prints trace messages generated by the firmware of 5th generation devices Dumps trace messages generated by the device hardware. Dumps device internal configuration registers. The dump file can be used by Mellanox Support for hardware troubleshooting. Displays the current multicast groups and flow steering rules configured in the device. Target users: Developers of Flow Steering aware applications. Dumps the current QP contexts and Work Queues of ConnectX family network adapter cards and Connect-IB adapter cards. Generates an i2c transaction using an mtusb usb to i2c adapter or using the device internal i2c compatible master Scans the i2c bus Routes the i2c bus of an externally managed InfiniscaleIV/SwitchX system to connect to the switch silicon. Reads the hardware temperature from Mellanox Technologies devices internal sensors and prints the reading in Celsius degrees. Corrupts the next transmitted packet from the ConnectX family network adapter cards and Connect- IB adapter cards. Windows FreeBSD Operating System Rev Mellanox Technologies 7

8 Table 2 - Mellanox Firmware Tools (MFT) Available Tools (Sheet 3 of 3) Category Tool Description Operating System Debug and Diagnostics Utilities mlxuptime mlxfwreset Detailed installation instructions along with complete descriptions of the various tools in the package can be found in the Mellanox Firmware Tools User s Manual. 1.2 Software Dependencies Prints Mellanox devices' up time and measured/configured core clock frequency (at Beta level) Load Firmware after firmware update on ISFU capable devices.(5th generation devices) Windows FreeBSD mlxmdio Used to read/write MDIO registers (Clause 45) on boards with externally managed PHY mlxreg This tool exposes supported access registers, and allows users to obtain information regarding the registers fields and attributes, and to set and get data with specific register. mstregdump Dumps device internal configuration data. mcra Reads/writes a single word from/to a device configuration register space mlxcables This tool reads/writes Mellanox cable registers and queries the cables info Windows FreeBSD mlxvpd Reads PCI device VPD Windows VMware ESXi Table 3 - MFT Software Dependencies Software Package Required Version Kernel sources OFED / MLNX_OFED 1, 2 Perl Python 3 lsusb 4 rpmbuild xz 5 Machine s kernel version or higher 5.6 or later 2.6 and above Windows Mellanox WinOF VPI and later 8 Mellanox Technologies Rev 4.6.0

9 Overview Table 3 - MFT Software Dependencies Software Package Required Version VMware ESXi Python 2.6 and above 1. OFED can be downloaded from Note that installing OFED is not required if you wish to install MFT without In-Band capabilities. 2. For the mst ib add command to run, one of the OFED packages ibutils or ibutils2 or infiniband-diags should be installed and available in the PATH. (For details on OFED installation, visit and under OFED.) 3. Required for the mlxmcg tool only. 4. Required for the mtusb device usage. 5. For creating UPMF (update package for Mellanox firmware) 6. WinOF is required only for In-Band access. The package can be downloaded from > Products > Software >InfiniBand /VPI Drivers >Windows SW Drivers. Rev Mellanox Technologies 9

10 1.3 Supported Operating Systems and Platforms MFT is supported on the following platforms:. Table 4 - Supported Operating Systems and Platforms OS Windows 7 Windows 8.1 Windows 10 Windows Server 2008 R2 Windows Server 2012 Windows Server 2012 R2 Windows Server 2016 WinPE 3.0 WinPE 4.0 WinPE 5.0 WinPE 5.1 VMware ESXi 5.5 Native VMware ESXi 5.5 VMK VMware ESXi 6.0 Native VMware ESXi 6.0 VMK VMware ESXi 6.5 Native FreeBSD 9.1-RELEASE FreeBSD 11.0-RELEASE FreeBSD 10.3-STABLE FreeBSD 12-CURRENT RHEL6.2 RHEL6.3 RHEL6.5 RHEL6.6 RHEL6.6 RHEL6.7 RHEL6.8 RHEL7.0 RHEL7.1 RHEL7.2 RHEL7.3 Debian 7.6 Arch x86/ x86/ x86/ x86/ amd64 amd64 amd64 amd64 PPC64 / PPC64 / PPC64 / PPC64 /PPC64/ PPC64LE [Power8] / PPC64/ PPC64LE [Power8]/ARM /PPC64/ PPC64LE [Power8] 10 Mellanox Technologies Rev 4.6.0

11 Overview Table 4 - Supported Operating Systems and Platforms OS Arch Debian 8.0 Debian 8.1 Debian 8.2 Debian 8.3 Fedora 19 Fedora 20 Fedora 21 / PPC64LE [Power 8] Fedora 22 PPC64LE [Power8] Fedora 23 / PPC64 [Power8]/ PPC64LE [Power8] Fedora 24 GAIA R77.30 GAIA R80 OEL 6.5 OEL 6.6 OEL 6.7 OEL 6.8 OEL 7.1 Sles10 SP3 Sles11 SP1 Sles11 SP2 Sles11 SP3 / PPC64 [Power 7] Sles11 SP4 / PPC64 Sles12 / PPC64LE [Power 8] Sles12SP1 / PPC64LE Sles12SP2 / PPC64LE XenServer6.5 XenServer7.0 WindRiver6.0 XenServer4.2 PowerKVM3.1.x PPC64LE Ubuntu Ubuntu / PPC64LE [Power 8] Ubuntu / PPC64LE [Power8] Ubuntu / PPC64LE [Power 8] Rev Mellanox Technologies 11

12 Table 4 - Supported Operating Systems and Platforms OS Arch Ubuntu / PPC64LE [Power8] Ubuntu / PPC64LE [Power 8] Ubuntu /PPC64LE [Power 8] Kernel.org Kernel.org 3.16 Kernel.org 3.17 Kernel.org 3.18 Kernel.org 3.19 Kernel.org Supported Flash Types MFT supports the following Flash types. Table 5 - Supported Flash Types Vendor Flash Family Tested P/N Micron M25Pxx M25P16 M25PXxx M25PX16 N25Qxxx N25Q032 Winbond W25QxxBV W25Q32BV Spansion S25FL11xx S25FL116K Atmel AT25DFxxx AT25DF Supported Mellanox ICs With respect to MFT, Mellanox IC devices are divided into two groups: Group I and Group II (4th generation and 5th generation, respectively). The ICs are listed in the following table: Table 6 - Mellanox IC Devices IC Group Group I/4th Generation ConnectX -3 ConnectX -3 Pro SwitchX SwitchX -2 IC Device 12 Mellanox Technologies Rev 4.6.0

13 Overview Table 6 - Mellanox IC Devices IC Group IC Device Group II/5th Generation Connect-IB Switch-IB Switch-IB 2 Spectrum ConnectX -4 ConnectX -4 Lx ConnectX -5 (at beta level) ConnectX -5 Ex (at beta level) Rev Mellanox Technologies 13

14 2 Changes and New Features in Rev Table 7 - Changes and New Features in Rev Component/ Tool Description Operating System Adapter Cards Added support for ConnectX-5/ConnectX-5 Ex adapter cards. mlxconfig mlxburn Note: ConnectX-5/ConnectX-5 Ex adapter cards are currently at Beta level. Added an option to query active (current) configurations in mlxconfig. Added new parameters in VPI settings configuration: XFI_MODE, PHY_TYPE, FORCE_MODE Added a new parameter to the PCI configuration NON_PREFETCHABLE_PF_BAR Added the ability to use mlxvpd to read the device VPD when using mlxburn. fwreset Added support for fwreset in PPC64 and PPC64LE platforms. For further information, please refer to the MFT User Manual., Windows, VMware ESXi 14 Mellanox Technologies Rev 4.6.0

15 Known Issues 3 Known Issues The following table provides a list of known issues and limitations in regards to this release of the Mellanox Firmware Tools. Table 8 - Known Issues and Limitations (Sheet 1 of 8) Internal Ref. Issue / Description: Tools that run in parallel on the same device may interrupt one another, and may cause the device to be in an undefined state. Workaround: Avoid running more than a single tool at a time with the same device Keywords: General Description: The MFT driver is not a native VMware driver, thus it can not take ownership of the PCI device in livefish mode / / / Workaround: N/A Keywords: General Description: The MFT driver is not a native VMware driver, thus it can access the Mellanox PCI device only after the MLNX-OFED ESXi driver marks it as accessible by vmkliunx drivers. Workaround: Install and load MLNXOFED ESXi driver (which is part of inbox drivers in ESXi 5.5 and newer) Keywords: General Description: MFT does not support MTUSB devices nor remote ETH and IB devices in FreeBSD and VMware ESXi. Workaround: N/A Keywords: General Description: PCI access in PowerPC machines is up to 10 times slower than other platforms, therefore, there are performance issues on all tools running on PowerPC machines. Workaround: N/A Keywords: General - Description: Running MFT tools on ConnectX-4/ConnectX-4 Lx on a Windows machine with PCI Gen1 or Gen2 causes machine to hang when Network adapted is disabled. For Example: Running mlxfwreset on ConnectX-4/ConnectX-4 Lx on the aforementioned server causes the machine to hang since disabling the network adapter is a part of the reset flow. Workaround: Enabled the network adapter before using any of the tools. For loading the firmware, reboot the server. Keywords: General Rev Mellanox Technologies 15

16 Table 8 - Known Issues and Limitations (Sheet 2 of 8) Internal Ref. Issue Description: Wrong localization settings in the operating system cause some tools to fail with error: locale::facet::_s_create_c_locale name not valid Workaround: Run: export LC_ALL=C Keywords: General Description: Aggressive killing of a tool that is locking the vendor specific semaphore (in ConnectX-4/ConnectX-4 Lx devices) will leave the semaphore locked, and any tool will get stuck waiting for semaphore. Workaround: Run: mcra -c <pci device> Keywords: General Description: When burning firmware with old MFT tools, the firmware version comparison might not work correctly and might return a prompt that the old firmware is newer. This is due to changes in firmware version format. Workaround: When prompted, press "y" to force update, or add the following flag to the command line: flint: -y --yes mlxburn: -force mlxfwmanager: -f --force Keywords: Firmware Burning Tools - Description: Firmware burn will fail due to invalid/old timestamp on ConnectX-4/ConnectX-4 Lx devices running these firmware versions: / , or / Workaround: Use MFT v4.1.0 to upgrade/downgrade the firmware version. Keywords: Firmware Burning Tools Description: On ConnectX-3/ConnectX-3 Pro adapter cards, when burning a new firmware image that does not contain an expansion ROM on a flash that contains firmware and expansion ROM, the burning tool will save the expansion ROM in the flash. Workaround: This issue can be avoided by: 1. Removing the ROM from the flash before the FW update process by running the following flint command: flint -d <dev> --allow_rom_change drom 2. Adding the --use_image_rom flag to the flint FW update command Keywords: Firmware Burning Tools Description: mlxfwmanager self-extractor tool will not work properly in ESXi machines if there is no mlx driver or MST installed on the machine. Workaround: Install the driver and the mst that matches the machine, then run the tool. Keywords: mlxfwmanager 16 Mellanox Technologies Rev 4.6.0

17 Known Issues Table 8 - Known Issues and Limitations (Sheet 3 of 8) Internal Ref. Issue / Description: mlxconfig allows setting port parameters for both ports from a single function on ConnectX-4. This poses a security issue since a certain physical function is able to change port parameters for all ports regardless of its designated port. Workaround: N/A Keywords: mlxconfig Description: mlxphyburn fails to query firmware version. Workaround: N/A Keywords: mlxphyburn Description: mlxmdio PHY address may differ when the read/write transactions are performed either directly by manipulating the hardware gateway or by asking the firmware. Workaround: Run with -g 0 flag (forcing gateway access). Keywords: mlxmdio Description: FreeBSD PCI access API is currently not supported. Workaround: N/A Keywords: mlxburn Description: The firmware update process in Connect-IB, Switch-IB, ConnectX -4, ConnectX -4 Lx and ConnectX -5 and may take up to two minutes. Workaround: N/A Description: The sg command on Connect-IB, Switch-IB, ConnectX -4, ConnectX -4 Lx and ConnectX -5 fails if the -override_cache_replacement flag is not used / Workaround: Set the GUIDs only when firmware is not active (driver is not loaded) by using the flag -override_cach_replacement Description: Attempting to access Remote/MTUSB device for parallel does not work well. Workaround: Avoid working in parallel through these interfaces Description: Attempt to burn firmware while specifying the -use_fw flag fails as firmware does not support write operation on this flash. Workaround: Do not use the -use_fw flag for burn operations Description: wqdump does not support WQEs receiving of 1024B on Connect-IB, ConnectX -4 and ConnectX -4 Lx. Workaround: N/A Keywords: wqdump Rev Mellanox Technologies 17

18 Table 8 - Known Issues and Limitations (Sheet 4 of 8) Internal Ref. Issue Description: mst ib add may fail when there is a device with zero system GUID in the fabric. The failure occurs when the mst ib add uses the ibnetdiscover tool. Workaround: Run 'mst ib add --discover-tool ibdiagnet' Description: When an IB subnet manager is not running in the fabric, some of the ports may be in INIT state. Devices that are accessed via these ports are added to the in-band device list even though they are inaccessible to in-band traffic. Workaround: Verify a subnet manager is running and that all the ports are in ACTIVE state Description: In Windows, mst remote devices added by the mst remote add command may still be present after uninstalling MFT. Workaround: If you still see old remote devices after installing a new WinMFT, you can either run mst restart or remove the devs directory manually (resides under the WinMFT install directory) Description: In Windows, mst ib add uses only ibnetdiscover to discover the cluster --discover-tool option is not used to choose the intended tool. The discover tool argument is intended only for parsing purpose, thus the topology file must be specified when using the --discover-tool. Workaround: N/A Description: When i2c-diolan-u2c driver is installed, it takes ownership over the mtusb device and prevents mst to load it, consequently, pre-vents access to MFT tools. Workaround: Stop the i2c-diolan-u2c driver Description: Accessing MST device in Windows machine remotely in parallel is not supported; only one connection at the same time is supported. Workaround: N/A Description: When there are multiple ConnectX-4 devices in Windows, the MST status shows them in a gap of 2 in indexes (even numbers). Workaround: N/A Description: If an unexpected shutdown occurs after running the firmware update package (UPMF) in Windows, 'mst status' may not show any devices when the machine comes up. Workaround: Run: sc delete mst64 or sc delete mst32 for cleanup. 18 Mellanox Technologies Rev 4.6.0

19 Known Issues Table 8 - Known Issues and Limitations (Sheet 5 of 8) Internal Ref. Issue Description: XenServer machine does not have an rpmbuild command which is used to recompile the kernel-mft source RPM in installation phase therefore, MFT installation fails. Workaround: 1. Prepare the kernel-mft RPM in the DDK server of the xenserver installed in your machine by running: " install.sh --rebuild-srpm 2. Install the kernel-mft binary RPM generated in stage one in the xenserver normal machine by: "rpm -i <kernel-mft binary RPM>" 3. Install MFT by: "install.sh --without-kernel" Keywords: Install Script Description: If the mcg tool is running while steering entries are added or removed from the device, the tool may display warnings in the following format: "mcg [0x1bff5].next points to non-existing mcg index 0x1b7f5" And the displayed data may be incorrect. Workaround: It is recommended to run the tool when the steering table is in a static mode. Keywords: mlxmcg Description: mlxuptime and mget_temp are not working against INBAND ConnectX-4/ ConnectX-4 Lx devices. Workaround: Work against other available devices interfaces such as PCI/remote. Keywords: mlxuptime Description: Terminating the tool when running it in MEM mode with CTRL+Z can cause issues to the device and to the firmware. Workaround: Do not send CTRL+Z to the tool. Keywords: mlxtrace Description: When running mlxtrace against in-band device, a ctrl+c is handled by the other library that is used by mlxtrace. Therefore, mlxtrace does not create all files needed. Workaround: N/A Keywords: mlxtrace Description: Downgrading WinMFT to an older version might corrupt the installation dir. Workaround: Remove MFT prior to installing any earlier versions of MFT. Keywords: WinMFT Installation Description: mlxfwreset is not supported in VMware. Workaround: N/A Keywords: mlxfwreset Description: When attempting to reset a pass-through device on a VM, the reset will fail because mlxfwreset fails to locate the PCI bridge for the specified device. Workaround: To load the firmware, reboot the machine Keywords: mlxfwreset Rev Mellanox Technologies 19

20 Table 8 - Known Issues and Limitations (Sheet 6 of 8) Internal Ref / / / Description: mlxfwreset is not functional is FreeBSD 9.1. Workaround: Reboot the machine after firmware upgrade. Keywords: mlxfwreset Issue Description: MFT is not functional if Advance Toolchain Runtime (at7.1) for PPC64LE is installed on the machine. Workaround: Export LD_PRELOAD="<path to the correct libc>" / For example: Export LD_PRELOAD="/lib/powerpc64le-linux-gnu/libc.so.6" Keywords: Advance Toolchain Runtime, PPC64LE Description: Segmentation fault may occur in fwtrace on RedHat 6.5 and 6.7 systems. Workaround: N/A Keywords: fwtrace Description: In order for mlxcables to work locally on the switch, the following switch firmware version must be installed on the switch: Switch-IB or higher Switch-IB or higher Spectrum or higher Workaround: N/A Keywords: mlxcables Description: When upgrading firmware from v to v12/ using mlxfwreset, Pass-Through in ConnectX-4 adapter cards might not function properly. In this case, such error message will appear: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio: Error: Failed to setup INTx fd: No such device T06:46: Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: Device initialization failed. Workaround: Reboot the server for the firmware upgrade changes to take effect. Keywords: Installation, firmware upgrade Description: Running mlxtrace/fwtrace with MEM mode on Connect-IB/ConnectX-4/ConnectX-4 Lx and ConnectX-5 devices is not supported in FreeBSD and VMware ESXi. Workaround: Use tracer mode FIFO (-m FIFO) / Keywords: mlxtrace/fwtrace Description: mlxreg does not support big registers (the maximum SMP MAD size is 64 bytes) when running it with an Inband device (through IB mads). Workaround: Run mlxreg locally with the PCI device Keywords: mlxreg/fwtrace 20 Mellanox Technologies Rev 4.6.0

21 Known Issues Table 8 - Known Issues and Limitations (Sheet 7 of 8) Internal Ref. Issue - Description: For mlxfwreset to work on PowerPC, the minimum firmware version that must be installed on the device is: ConnectX-5: ConnectX-4: ConnectX-4 Lx: ConnectIB: Workaround: N/A Keywords: mlxfwreset Description: Running mlxfwreset on an ARM server with Ubuntu 16.04, may result in machine hanging. Workaround: Reboot the server in order to load the firmware Keywords: mlxfwreset Description: MFT tools do not function properly with the interactive prompt on Windows Server 2016: Nano server. Workaround: Use the -y/--yes, -n/--no flags with the relevant tools to allow non interactive execution. Keywords: Windows Server 2016: Nano server Description: mst.exe is not functional in Windows Server 2016: Nano Server. Workaround: Use mdevices.exe to get list of available devices Keywords: Windows Server 2016: Nano server, mst Description: mlxfwreset is not functional on MultiHost and Socket Direct NICs. Workaround: To load the firmware, shut down all related hosts and then power up. Keywords: mlxfwreset Description: When running the mlxconfig query after mlxconfig reset, the mlxconfig query does not show the correct configuration that should be loaded upon the next boot. Workaround: To view the default configuration (under the Default column) run: mlxconfig -d <device> -e query. Keywords: mlxconfig Description: Running mlxfwreset on Connect-IB on PowerPC setup on SLES OS, may result in mlxfwreset failure. Workaround: To reload the firmware, reboot the server. Keywords: mlxfwreset Description: Image generation fails when generating a ConnectX-5 image on FreeBSD12- CURRENT. Workaround: N/A Keywords: ConnectX-5, image generation, FreeBSD12-CURRENT Rev Mellanox Technologies 21

22 Table 8 - Known Issues and Limitations (Sheet 8 of 8) Internal Ref. Issue Description: Upon each boot, for the first module that either has no signature or its key is not in the kernel key ring, the following messages will be presented: mst_pci: module verification failed "mst_pci : module verification failed: signature and/or required key missing - tainting kernel Note: These messages cannot be eliminated and can be easily missed as they are not seen upon unloading and reloading the kernel module until its reboot. Workaround: N/A _pci, kernel module Description: mlxfwreset may fail to reset the device on Ubuntu PPC64LE systems when multiple kernels are installed. Workaround: Use systems with a single kernel installed. Keywords: kernel module, mlxfwreset, Ubuntu PPC64LE 22 Mellanox Technologies Rev 4.6.0

23 Bug Fixes History 4 Bug Fixes History Table 9 lists the history of bugs fixed Table 9 - Bug Fixes History (Sheet 1 of 16) Internal Ref / / / Issue Description: Occasionally, MFT tools (driver mode) do not function after running mlxfwreset in PowerPC machines. Keywords: mlxfwreset Discovered in Release: Fixed in Release: Description: mlxburn fails to read VPD on machines where the device shares the same B:D:F address with another PCI device on different PCI domains Keywords: mlxburn Discovered in Release: Fixed in Release: Description: Reading the VPD using the -vpd_rw flag or programing the VPD may take up to 5 mins / / Keywords: mlxburn Discovered in Release: Fixed in Release: Description: mlxfwreset disables and enables all Mellanox devices Network Interfaces when resetting the firmware on a device that at least one of its network interfaces is up. Keywords: mlxfwreset Discovered in Release: Fixed in Release: Description: Running mlxfwreset against OEM devices may enter the device to a undefined state. Keywords: mlxfwreset Discovered in Release: Fixed in Release: Description: PPTT and PPRT registers are not supported in switches. Keywords: mlxreg Discovered in Release: Fixed in Release: Rev Mellanox Technologies 23

24 Table 9 - Bug Fixes History (Sheet 2 of 16) Internal Ref / Description: Fixed an issue causing the mlxcables tool to show wrong temperature value when querying the cable several times in loop. Keywords: mlxcables Discovered in Release: Fixed in Release: Description: In /Windows, updating firmware on an unmanaged InfiniScale IV switch by Inband fails with the following error: MFE_SEM_LOCKED Keywords: Firmware Burning Tools Discovered in Release: Fixed in Release: Description: Added fwtrace and mlxfwreset to the MFT package for PPC. Keywords: General Discovered in Release: Fixed in Release: Description: Fixed an issue that caused the target machine to hang when running mlxtrace over an mtusb connected to an HCA/NIC in a Windows target machine. In this mode, mlxtrace may cause memory corruption on the target machine which causes it to hang. Keywords: mlxtrace Discovered in Release: Fixed in Release: Description: Fixed an issue causing read/write corruption and yields unexpected behavior when running mlxfwmanager_pci with MFT tools in parallel. Keywords: mlxfwmanager Discovered in Release: Fixed in Release: Description: Fixed an issue causing mlxfwreset to fail due to some missing files in ARM machines. Keywords: mlxfwreset Discovered in Release: Fixed in Release: Description: Added mlxfwmanager as part of the MFT package intended to FreeBSD 9.1. Keywords: mlxfwmanager. Discovered in Release: Fixed in Release: Issue 24 Mellanox Technologies Rev 4.6.0

25 Bug Fixes History Table 9 - Bug Fixes History (Sheet 3 of 16) Internal Ref Description: Fixed functionality issues of query/set VPD flags in mlxburn in ESXi VMware. Keywords: mlxburn Discovered in Release: Fixed in Release: Description: Fixed cpldupdate calibration option (-c flag) functionality. Keywords: cpldupdate Discovered in Release: Fixed in Release: Description: Fixed an issue which showed both Base MAC and Base GUID when running mlxfwmanager query on ConnectX-4 devices, even if the two ports were configured to be ETH only, or IB only / / Keywords: mlxfwmanager Discovered in Release: Fixed in Release: Description: Fixed an issue preventing mlxfwreset from working in when MLNX_OFED or an inbox driver was not installed. Keywords: mlxfwreset Discovered in Release: Fixed in Release: Issue Description: Fixed an issue which caused flint brom/drom operations to fail when using 5th Generation devices firmware. Discovered in Release: Fixed in Release: Description: Performing mlxfwreset on a server with SR-IOV enabled corrupts the virtual functions Keywords: mlxfwreset Discovered in Release: Fixed in Release: Description: Resetting the firmware using mlxfwreset in a machine where the Mellanox driver is Inbox (not MLNX_OFED) might cause the device to hang Keywords: mlxfwreset Discovered in Release: Fixed in Release: Rev Mellanox Technologies 25

26 Table 9 - Bug Fixes History (Sheet 4 of 16) Internal Ref. Issue Description: MFT package does not include support for ConnectX-4 and ConnectX-4 Lx devices for VMware Keywords: General Discovered in Release: Fixed in Release: Description: ConnectX-4 and ConnectX-4 Lx devices have a physical function per port, for example: 01:00.0 and 01:00.1. MFT creates an MST device for the first function only. Therefore, MFT can run only against the first function Keywords: General Discovered in Release: Fixed in Release: Description: On PowerPC 8, mst status -v does not show any information on the multiple functions of the device. For example, in ConnectX-4 devices, each port has a function of its own by default, but it is not shown in mst status -v Discovered in Release: Fixed in Release: Description: Setting one of the ports to ETH and the other to IB using mlxconfig may cause the firmware to crash. This was fixed in ConnectX-4 firmware v Keywords: mlxconfig Discovered in Release: Fixed in Release: Description: HCR tools are not supported in FreeBSD Keywords: mlxconfig/mlxphyburn Discovered in Release: Fixed in Release: Description: pci_cr interface is not supported in VMware ESXi 2015 Keywords: General Discovered in Release: Fixed in Release: Description: Running online updates/downloads in parallel may fail Keywords: mlxfwmanager Discovered in Release: Fixed in Release: Mellanox Technologies Rev 4.6.0

27 Bug Fixes History Table 9 - Bug Fixes History (Sheet 5 of 16) Internal Ref. Issue Description: Dumping eqe on Connect-IB and ConnectX-4 adapters fails Keywords: wqdump Discovered in Release: Fixed in Release: Description: Reset configuration is not supported in mlxconfig for ConnectX-4 Keywords: mlxconfig Discovered in Release: Fixed in Release: Description: UPMF Generation is not supported in VMware ESXi and FreeBSD Keywords: General Discovered in Release: Fixed in Release: Description: mlx_mfa_gen can not create MFA file under directory with whitespace Keywords: mlxfwmanager Discovered in Release: Fixed in Release: Description: flint sg/sv/set_vsd commands with Connect-IB firmware or newer corrupt the firmware image Discovered in Release: Fixed in Release: Description: Running burning tools (for burn/query) and restarting the driver or machine boot at the same time may cause the machine to hang Keywords: Burning tools (mlxfwmanager, flint and mlxburn) Discovered in Release: Fixed in Release: Description: mlxtrace activates a hardware component which may lead to decreasing the packet rate of the HCA Keywords: mlxtrace Discovered in Release: Fixed in Release: Rev Mellanox Technologies 27

28 Table 9 - Bug Fixes History (Sheet 6 of 16) Internal Ref. Issue Description: If a burning process is killed forcefully (such as: by kill -9 in ), subsequent runs of burning tools will not function as they require hardware resources cleanup Keywords: Burning tools (mlxfwmanager, flint and mlxburn) Discovered in Release: Fixed in Release: Description: mst status -v does not show MLNX_OFED device name (mlx4/mlx5) in XEN server machines. Hence, the MFT tools cannot be run on these devices. Also, mlxtrace/ fwtrace on Connect-IB in MEM mode cannot work, since it is using these devices Discovered in Release: Fixed in Release: Description: mlx_mfa_gen fails to calculate CRC when MFA's directory name includes spaces Keywords: mlxfwmanager Discovered in Release: Fixed in Release: Description: mlxfwmanger_pci fails to detect devices in virtual machines of RedHat KVM. <Where the pci address of the device has dev.func!=00.0> Keywords: mlxfwmanager_pci Discovered in Release: Fixed in Release: Description: Stopping/restarting mst service while one tool is running may cause some issues with the device or driver or the tool itself Keywords: Discovered in Release: Fixed in Release: Description: mft_uninstall failed in some Ubuntu machines Keywords: mft_uninstall.sh Discovered in Release: Fixed in Release: Description: mlxfwreset failed due to a parsing issue on some machines where the Express capability register address contains hex digit (such as: b0) Keywords: mlxfwreset Discovered in Release: Fixed in Release: Mellanox Technologies Rev 4.6.0

29 Bug Fixes History Table 9 - Bug Fixes History (Sheet 7 of 16) Internal Ref. Issue Description: On some Connect-IB board types driver failed to load after loading new firmware using mlxfwreset Keywords: mlxfwmanager Discovered in Release: Fixed in Release: Description: Kernel installation failed in fbk13 Operating System Keywords: Installation Discovered in Release: Fixed in Release: Description: MFT installation failed in PPC64LE when TCL is not installed there Keywords: Install script and mlxburn Discovered in Release: Fixed in Release: Description: Burning tools may cause machine reboots in some ConnectX cards Keywords: Burning tools Discovered in Release: Fixed in Release: Description: Updating a non-failsafe image is not supported Keywords: Mlxfwmanager Discovered in Release: Fixed in Release: Description: Killing an WinMFT process may interrupt subsequent runs of WinMFT tools Keywords: Mlxfwmanager Discovered in Release: Fixed in Release: Description: mlxfwmanager_pci does not work in VMware Keywords: Mlxfwmanager Discovered in Release: Fixed in Release: Description: MFT supports only one connected MTUSB-1 device at a time Keywords: Discovered in Release: 2.7.2b Fixed in Release: Rev Mellanox Technologies 29

30 Table 9 - Bug Fixes History (Sheet 8 of 16) Internal Ref. mlxuptime Issue Description: The displayed configured frequency may present wrong results Keywords: mlxuptime Discovered in Release: Fixed in Release: Description: mst remote feature does not work when the client machine is windows and the server machine is Discovered in Release: Fixed in Release: Description: Devices in livefish mode in FreeBSD does not appear in mst status Discovered in Release: Fixed in Release: Description: Cannot read vpd when accessing via PCI device in the format: domain:bus:dev.fn Keywords: mlxburn Discovered in Release: Fixed in Release: Description: First flint run may fail after Ctrl+C Discovered in Release: Fixed in Release: Description: Flint Connect-IB burn corrupts the flash when burning a verbatim full flash dump as the firmware image Discovered in Release: Fixed in Release: Description: Flint Connect-IB burning using the -vsd flag is not functional Discovered in Release: Fixed in Release: Description: Occasionally, CTRL+C causes flint to hang in WinPE OS Discovered in Release: 2.7.2b Fixed in Release: Mellanox Technologies Rev 4.6.0

31 Bug Fixes History Table 9 - Bug Fixes History (Sheet 9 of 16) Internal Ref. Issue - Description: Unexpected behavior when running swreset on flint Discovered in Release: 2.7.2b Fixed in Release: Description: CTRL+C does not clear semaphores Discovered in Release: 2.7.2b Fixed in Release: Description: Connect-IB burning: Flint supports VSD of up to 128 chars Discovered in Release: Fixed in Release: Description: wbne command causes endiannes swap Discovered in Release: Fixed in Release: Description: On Windows Power Shell Ctrl+C does not function properly Discovered in Release: Fixed in Release: Description: On FreeBSD, burning new firmware on a device with corrupt empty firmware does not work Discovered in Release: Fixed in Release: Description: Occasionally, if a node has a LID set to 0, the mst ib add --use-ibdr command will not add that nodes to the mst device list Discovered in Release: Fixed in Release: Description: On Windows, mst may not show any devices Discovered in Release: Fixed in Release: Rev Mellanox Technologies 31

32 Table 9 - Bug Fixes History (Sheet 10 of 16) Internal Ref. - Description: mlxmcg fails on PPC Keywords: mlxmcg (formerly mcg) Discovered in Release: Fixed in Release: Issue - Description: On Windows Ctrl+C does not exit gracefully Keywords: itrace Discovered in Release: Fixed in Release: Description: On Windows Power Shell MFT might not work on mst devices that contain a comma Keywords: Discovered in Release: Fixed in Release: Description: On Windows x64, some tools are 32 bit binaries Keywords: Discovered in Release: Fixed in Release: Description: The mst cr device is not functional on VMware VMware ESXi. When running it, the following message is received: You cannot access the HW configurations by mt4099_pci_cr0 device in ESXi Discovered in Release: Fixed in Release: Description: Running the sg (set guids) command on a striped image file containing a large expansion rom image may fail Discovered in Release: Fixed in Release: Description: mst status does not show the mtusb-1 when adding in-band devices Discovered in Release: Fixed in Release: Mellanox Technologies Rev 4.6.0

33 Bug Fixes History Table 9 - Bug Fixes History (Sheet 11 of 16) Internal Ref. Issue - Description: mst shows all devices when connected to Flex10 remotely Discovered in Release: Fixed in Release: Description: mst does not display a message when there is no HCA device connected to the machine Discovered in Release: Fixed in Release: Description: mst ib add --use-ibdr may not add all nodes in a cluster when the ports are inactive Discovered in Release: Fixed in Release: Description: mtusb is not supported in Windows 2012 Discovered in Release: Fixed in Release: Description: Running the itrace tool when the driver is not loaded or when the driver failed to load may cause the server to crash Keywords: itrace Discovered in Release: Fixed in Release: Description: Setting the GUIDs on an image file which was generated using the - exp_rom and -striped_image flags fails when the expansion rom size is larger than 400KB. Current rom sizes are much smaller, thus it is not expected to cause an actual issue Discovered in Release: Fixed in Release: Description: mlxtrace not functional when using the PCI format device against Connect- IB Keywords: mlxtrace Discovered in Release: Fixed in Release: Rev Mellanox Technologies 33

34 Table 9 - Bug Fixes History (Sheet 12 of 16) Internal Ref. Issue Description: CTRL-C stops wqdump but does not clear (release) semaphores Keywords: wqdump Discovered in Release: Fixed in Release: Description: Running wqdump with -ignore ignores only the QPC gateway lock but does not ignore the OB gateway Keywords: wqdump Discovered in Release: Fixed in Release: Description: Setting an empty keyword is not reported as an error, but will prevernt further using of the vpd_set_keyword flag Setting a keyword value longer than 255 characters is not reported as an error, but may corrupted the vpd Keywords: mlxburn Discovered in Release: Fixed in Release: Description: The flag "-fw_dir" is not supported when burning ConnectX -3 Pro adapter card Keywords: mlxburn Discovered in Release: Fixed in Release: Description: When a read/write operation over mtusb/remote device fails while running flint, flint may crash, hence not handling the error well Discovered in Release: Fixed in Release: Description: When trying to burn a firmware+rom that contains a unified product version to a device where the firmware has no product version, the flint tool allows keeping the device ROM instead of burning the new firmware+rom as is Discovered in Release: Fixed in Release: Mellanox Technologies Rev 4.6.0

35 Bug Fixes History Table 9 - Bug Fixes History (Sheet 13 of 16) Internal Ref Description: Access the trace messages area in host memory may fail when using Mellanox HCA cards without on-board memory (MemFree) running on Red Hat Fedora and OEL OSs. Consequently, the application may crash or no trace messages will be printed Keywords: itrace Discovered in Release: Fixed in Release: Description: Activating counter gateways in ConnectX -3 via an inband device causes firmware to hang (dead lock) Keywords: mlxdump Discovered in Release: Fixed in Release: / / Issue Description: Installing MFT with a different prefix than /usr, causes mlxmcg, fwtrace and mstdump to not work. On Fedora OS, in addition to the above tools, mst and mlxburn do not work either Keywords: Installation Discovered in Release: Fixed in Release: Description: Running mget_temp against ConnectX -3 Pro even after initializing the diode results in high temperature (e.g. 120) Keywords: mget_temp Discovered in Release: Fixed in Release: Description: On certain boards where the external oscillator is connected only to one port, the other port's thermal sensor will not function Keywords: mget_temp Discovered in Release: Fixed in Release: Description: On SwitchX-A0, the YU-sensor reports the wrong temperature Keywords: mget_temp Discovered in Release: Fixed in Release: Description: Installing MFT with a different prefix than /usr, causes mlxmcg to not work Keywords: mlxmcg (formerly mcg) Discovered in Release: Fixed in Release: Rev Mellanox Technologies 35

36 Table 9 - Bug Fixes History (Sheet 14 of 16) Internal Ref. Issue Description: Occasionally, the fwtrace executable may stop working and display the following output: Cannot open self /usr/bin/fwtrace or archive /usr/bin/fwtrace.pkg Keywords: fwtrace Discovered in Release: Fixed in Release: Description: wqdump uses certain hardware gateways which are used by the firmware. Thus when it accesses the device via the firmware and owns the hardware gateways, it causes issues for the firmware and does not work Keywords: wqdump Discovered in Release: Fixed in Release: Description: MFT cannot be installed in a path different than the default /usr Keywords: Install script Discovered in Release: 2.7.1a Fixed in Release: Description: The installation script fails if OFED v1.4 or older is installed in the machine as it tries to search for non-existing RPMs Keywords: Install script Discovered in Release: 2.7.1a Fixed in Release: Description: After firmware upgrade or downgrade, the externally managed SwitchX-A1 should be rebooted through power-cycle and not via flint "swreset" command Discovered in Release: 2.7.1a Fixed in Release: Description: The flag "-fw_dir" is not supported when burning ConnectX-3 A1 adapter card Keywords: mlxburn Discovered in Release: 2.7.1a Fixed in Release: Description: The uninstall operation leaves device files under /dev/mst Discovered in Release: 2.7.1a Fixed in Release: Mellanox Technologies Rev 4.6.0

37 Bug Fixes History Table 9 - Bug Fixes History (Sheet 15 of 16) Internal Ref. Issue - Description: The tool shows the rules of port 1 in port2 line and vice versa Keywords: mlxmcg (formerly mcg) Discovered in Release: 2.7.1a Fixed in Release: Description: Running mstdump on a ConnectX -3 device may cause the firmware to hang dump Discovered in Release: Fixed in Release: 2.7.1a - Description: Accessing the SwitchX flash by MFT has a bug, while reading on PPC64 platform, therefore the SwitchX cannot be updated via the PPC64 machine Discovered in Release: Fixed in Release: Description: The flag -fw_dir is not supported when burning SwitchX devices Keywords: mlxburn Discovered in Release: 2.7.1a Fixed in Release: Description: Running mstdump on a SwitchX device may cause it to malfunction dump Discovered in Release: 2.7.1a Fixed in Release: Description: Burning the firmware into ConnectX -3 A1 adapter using the MST CR device resulted in failure Discovered in Release: Fixed in Release: 2.7.0a - Description: In-band access is not supported in Windows MFT Discovered in Release: Fixed in Release: Description: No MTUSB-1 support for 64-bit architecture in Windows MFT Keywords: Discovered in Release: Fixed in Release: Rev Mellanox Technologies 37