Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes Versions: ESXi 5.5 driver FCoE: 11.2.1153.13 NIC: 11.2.1149.0 iscsi: 11.2.1153.2 ESXi 6.0 driver FCoE: 11.2.1153.13 NIC: 11.2.1149.0 iscsi: 11.2.1147.5 ESXi 6.5 driver FCoE: 11.2.1153.13 NIC: 11.2.1149.0 iscsi: 11.2.1152.0 Date: January 30, 2017 Purpose and Contact Information These release notes describe the new features, resolved issues, known issues, and technical tips associated with these Emulex drivers for VMware releases. For the latest product documentation, go to www.broadcom.com. If you have questions or require additional information, contact an authorized Broadcom technical support representative at ccx-tech.support@broadcom.com or request assistance online at https://oemsupportportal.emulex.com/web2tech/ccx.html. New Features 1. Beginning with software release 11.2, LightPulse adapters and OneConnect adapters have independent software kits. Special instructions and considerations for using the driver kits for LightPulse and OneConnect adapters are provided in the Emulex Software Kit Migration User Guide. Refer to that document before updating existing drivers and applications to the drivers and applications in release 11.2. 2. The following operating system does not support new features in this release: VMware ESXi 5.5 3. The following operating system is no longer supported: VMware ESXi 5.1 Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 1
Resolved Issues 1. This version resolves an issue in which Emulex network adapter interfaces were configured in NIC Teaming mode, one of the interfaces went down, and a long failover time resulted in packet loss. 2. An issue in which the MBrd value in iostats displayed 0 when traffic flow was available is resolved. 3. This version resolves an issue in which bonding created on virtual function (VF) interfaces failed to switch the traffic to standby when the active physical function (PF) interface link was down. 4. A port on which setportenabled was set to 0 is no longer reported as operational. Known Issues All Versions, Known Issues 1. Link Aggregation Control Protocol (LACP) cannot be used on the same port as FCoE or iscsi. 2. The adapter model name and description do not match the vendor brand name field. The VMware vsphere client only shows adapter family model names with a single port. For example, it shows the OneConnect OCe11102 universal converged network adapter (UCNA) as the OneConnect OCe11100 10 Gigabit Ethernet (GbE), FCoE UCNA. This issue occurs with the native tool lspci as well. The vsphere client gets the adapter model name and description from the XML package files installed with the operating system, or with an out-of-box kit, rather than from the adapter vital product data (VPD). Other native tools work in a similar manner. To show all available KV instances, use the following command: /usr/lib/vmware/vmkmgmt_keyval/vmkmgmt_keyval -d 3. On an Emulex OCe14000-series adapter, if Universal Multichannel (UMC) is enabled and the maximum bandwidth is set using the PXESelect utility, the iscsiselect utility, or the Emulex OneCommand CNA Manager application, the I/O does not adhere to the set value. 4. A restriction exists in assigning data center bridging (DCB) priorities to priority groups. Although eight priority groups are available to which priorities can be assigned, you can to assign priorities and bandwidths to only two of the priority groups. You must assign the FCoE or iscsi priority to one priority group, and you must assign the other seven (NIC) priorities to the other priority group. Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 2
Note: If you are using a data center bridging exchange (DCBX)-enabled switch to configure the priority groups, configure it for only two priority groups to work correctly with the adapter. 5. Concurrent firmware upgrades performed on the same ESXi host might not be fully copied. If a firmware upgrade is attempted using the HTTP protocol, the firmware file is copied to the /filesystem of the ESXi system. This file is deleted after the firmware upgrade is complete. Therefore, if concurrent firmware upgrades are performed on the same ESXi host, due to space constraints, the firmware file might not be copied fully, and the firmware upgrades might fail with an error. 6. In ESXi 6.0 and 6.5, on an Emulex OCe14102 adapter with Virtual Extensible local area network (VXLAN), IPv6 transmit traffic goes down when TCP segmentation offloading (TSO) is disabled on both the hypervisor as well as on the virtual machine (VM). This occurs when using RHEL 6.6 or earlier as a guest operating system. Use RHEL 6.7 or later versions as a guest operating system, or disable TSO on the hypervisor and the VM. ESXi 5.5 Known Issues 1. In INT-X mode with UMC enabled, an ESXi NIC native driver reload might take up to 20 minutes to load. 2. On an OCe14000-series adapter, VXLAN performance might require several minutes to reach the line rate with 30 VMs. 3. Firmware netdump does not work properly on Emulex OCe11100-series adapters. 4. The ESXi 5.5 NIC driver has limited support for extended Single Root input/output virtualization (SR-IOV) support options. ESXi 5.5 has limited support for extended configuration networking options of a virtual function that is assigned to a virtual machine with compatibility for ESXi 5.5 or later. Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 3
Supported extended SR-IOV options Enabling Virtual Guest Tagging (VGT) Enabling VLAN Switch Tagging (VST) mode Extended SR-IOV option limitations Cannot change the size of the maximum transmission unit (MTU) (cannot enable jumbo frames) Cannot accept or drop incoming frames for a new address with the MAC address change option Cannot enable global promiscuous mode for virtual machine network adapters For more information regarding supported networking configurations, refer to the Emulex Drivers for VMware ESXi for OneConnect Adapters User Guide and the support section of the VMware web site. 5. Connectivity is lost after you disable the netqueue feature in the ESXi host. The native driver framework of the vmkernel has an issue in the netqueues disabled path that causes transmit packets to be sent to the elxnet driver with an invalid transmit (Tx) queue ID. This situation causes the elxnet driver to drop the Tx packets that are marked with an invalid TxQID. Keep the netqueues feature of the ESXi host enabled, but load the elxnet driver in legacy interrupt mode. This action results in creating a single net queue. esxcli system module parameters set -p msix=0 -m elxnet 6. Disabling MultiQueue (MQ) might seriously impact performance of the LPFC driver. For this release, do not disable MQ. 7. The vport NDLP state does not transition to unused after receiving an unsolicited port login (PLOGI) from an initiator. 8. An errant NULL print string write to console log might occur. 9. A system hang might occur on servers with more than eight CPU sockets. The driver and adapter must be installed on a server with eight CPU sockets or fewer. 10. Direct-attached configurations in a point-to-point topology do not connect. Use the driver default topology (loop). Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 4
11. Firmware netdump fails when NIC I/O is running on the same uplink that is configured for the netdump feature. 12. Elxnet network statistics for SR-IOV VFs are not working properly. PFs on OneConnect adapters do not support retrieval of VF network statistics from the PF interface. 13. For OCe14000-series adapters, driver parameter changes might not take effect if the server is power-cycled immediately after the change. Use the reboot operation instead of cycling the power. 14. VXLAN performance is degraded in ESX 5.5. The system might require several minutes to reach the line rate with multiple VMs. The more VMs, the longer it takes to reach line rate. Disable VXLAN offloads. 15. I/O does not resume on vports after the vport is disabled and then re-enabled. When virtual ports on a single physical port are set to round robin, and they are disabled and re-enabled one at a time, I/O does not resume properly on the re-enabled vports. I/O continues on the physical port. 16. An Emulex port is down while running the change MTU test case with non-vlan TCP IPv4 traffic. The port does not receive an interrupt. 17. The server hangs when VMs with multiple VFs attached are powered on and then powered off. ESXi 6.0 Known Issues 1. The VMware Update Manager (VUM) mechanism might fail for non-iovp elxnet drivers. This is a known vmkernel issue with ESXi 6.0. Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 5
2. Issues with the load-balancing policy Route Based on Physical NIC Load with ESXi 6.0 exist where failover might not happen at all. Use ESXi 6.0 U1 or newer VMkernel builds. 3. If multiple VMs with attached VFs are powered on for a second time, the power-on fails. ESXi 6.5 Known Issues No issues reported. Technical Tips 1. A max_multiq parameter has been added to the ESXi 5.5 and ESXi 6.0 native mode FCoE drivers. This parameter determines how many I/O channels (MSI-X vector/eq/cq tuples) are used in parallel to Fibre Channel Protocol (FCP) 10. The disable_mq and fcp_io_channels driver parameters have been removed from the ESXi 5.5 and ESXi 6.0 native mode FCoE drivers. For more information, refer to the Emulex Drivers for VMware ESXi for OneConnect Adapters User Guide. 2. On the Emulex OCe14000-series adapters, teaming of physical functions from the same port in Multichannel mode might result in lost connectivity. Partitions belonging to the same port must be assigned to different vswitches (vnetwork Standard Switch [vswitch/vss] or VNetwork Distributed Switch [dvswitch/vds]). By design, the Emulex OCe14000-series adapters do not allow teaming from the NIC partitions on the same port. On Emulex OCe14000-series adapters configured in Multichannel Mode (this includes UMC and Flex modes), no more than one partition from a port can be assigned to a particular vswitch/vss or dvswitch/vds). The Emulex OCe14000-series adapters do not allow this assignment because: Replication of traffic results in excess usage of Peripheral Component Interconnect (PCI) bandwidth and lowers throughput. If a port on an adapter is in a bad state or the link is down, all the partitions of that port also go down. Teaming partitions on the same port do not provide the redundancy expected from teaming. Example of a vmkernel.log message that might be seen in ESXi 5.5 and 6.0 (elxnet driver) follows. WARNING: elxnet: elxnet_applyqueuemacfilter:3438: vmnicx: Failed to add mac.. qidval:0, status (0x1) WARNING: elxnet: elxnet_mcccomplprocess:1105: 0000:00N:00.M: Mailbox/MCC command opcode 59-1 failed:status 1-22 Example of a vmkernel.log message that may be seen by in ESXi 5.1 (be2net driver): Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 6
Mailbox command opcode = 0x3b on pci function number 0x4 in subsystem = 0x1 Failed with status = 0x1, additional status=0x16 Command to apply MAC address filter failed 3. The server must be in ESXi maintenance mode to safely change the firmware in flash memory. For operations, such as firmware download, bring the server into ESXi maintenance mode prior to flashing the firmware. 4. Install an offline bundle locally on an ESXi server using the esxcli command. Install an offline bundle on a VMware server locally using the esxcli command. [root@testmachine ~]# esxcli software vib install --maintenance-mode -d <offline-bundle.zip> Where <offline-bundle.zip> is the file name of the offline bundle to be installed. 5. Throughput performance degrades on block sizes greater than 64 KB on Emulex adapters. This issue is fixed in this driver release, but you must set this fix and reboot the system as an administrator. There is one prerequisite, the ESXi operating system release must be ESXi 5.1 GA - build #799733 or higher. To engage this fix, enter the follow commands: esxcli system module parameter set -p lpfc_dma_boundary -m lpfc820 reboot 6. The default VMware driver settings might not be appropriate for optimal performance in all scenarios. If performance appears to be lower than expected, several driver and system settings can be modified to improve performance. Refer to the server documentation to determine the correct system settings and the optimal memory and processor configuration. Along with proper hardware configuration, some driver settings can be modified to improve performance. The following are some supported settings to examine while tuning for better performance. Refer to the Emulex Drivers for VMware ESXi for OneConnect Adapters User Guide for an explanation of the available settings. Use the vmxnet 3 adapter type for all guest operating system interfaces. Install VMware tools for all guest operating systems. 7. Using the Emulex esxcli.vib add on provides extended statistics and complete VXLAN statistics viewing. The vsish command is limited to 4 kb size output, and it is not useful for viewing statistics. 8. In NIC teaming scenarios, when failback is set to no, have at least one standby vmnic for it to work properly. 9. For ESXi hosts, the Emulex OneCapture utility should be used to retrieve dumps instead of the Emulex OneCommand CNA Manager application remote dump. For more information on the OneCapture utility, refer to the Emulex OneCapture for OneConnect Adapters User Guide. Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 7
10. The NIC driver can be configured using the vsphere Client. ESXi 5.5 has full support for configuration networking options of a virtual function that is assigned to a virtual machine with compatibility for ESXi 5.1 or later. The Emulex Drivers for VMware ESXi for OneConnect Adapters User Guide provides instructions for default VLAN tagging, VGT, and configuring a static MAC for a virtual function using the ESXi driver. You can also perform these configuration tasks using the vsphere Client. For more information regarding the vsphere Client, refer to the support section of the VMware web site. For product information and a complete list of distributors, please go to our web site: www.broadcom.com. Broadcom, the pulse logo, Connect everything, Avago Technologies, Avago, the A logo, Emulex, LightPulse, OneCapture, OneCommand, and OneConnect are among the trademarks of Broadcom and/or its affiliates in the United States, certain other countries and/or the EU. Copyright 2016 2017 Broadcom. All Rights Reserved. The term "Broadcom" refers to Broadcom Limited and/or its subsidiaries. For more information, please visit www.broadcom.com. Broadcom reserves the right to make changes without further notice to any products or data herein to improve reliability, function, or design. Information furnished by Broadcom is believed to be accurate and reliable. However, Broadcom does not assume any liability arising out of the application or use of this information, nor the application or use of any product or circuit described herein, neither does it convey any license under its patent rights nor the rights of others. Emulex Drivers for VMware ESXi for OneConnect Adapters Release Notes 8