Mellanox Innova IPsec Ethernet Adapter Card Software and Firmware Bundle Release Notes

Mellanox Innova IPsec Ethernet Adapter Card Software and Firmware Bundle Release Notes www.mellanox.com Mellanox Technologies

NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT ( PRODUCT(S) ) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS-IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale, CA 94085 U.S.A. www.mellanox.com Tel: (408) 970-3400 Fax: (408) 970-3403 Copyright 2018. Mellanox Technologies Ltd. All Rights Reserved. Mellanox, Mellanox logo, Accelio, BridgeX, CloudX logo, CompustorX, Connect-IB, ConnectX, CoolBox, CORE-Direct, EZchip, EZchip logo, EZappliance, EZdesign, EZdriver, EZsystem, GPUDirect, InfiniHost, InfiniBridge, InfiniScale, Kotura, Kotura logo, Mellanox CloudRack, Mellanox CloudXMellanox, Mellanox Federal Systems, Mellanox HostDirect, Mellanox Multi-Host, Mellanox Open Ethernet, Mellanox OpenCloud, Mellanox OpenCloud Logo, Mellanox PeerDirect, Mellanox ScalableHPC, Mellanox StorageX, Mellanox TuneX, Mellanox Connect Accelerate Outperform logo, Mellanox Virtual Modular Switch, MetroDX, MetroX, MLNX-OS, NP-1c, NP-2, NP-3, NPS, Open Ethernet logo, PhyX, PlatformX, PSIPHY, SiPhy, StoreX, SwitchX, Tilera, Tilera logo, TestX, TuneX, The Generation of Open Ethernet logo, UFM, Unbreakable Link, Virtual Protocol Interconnect, Voltaire and Voltaire logo are registered trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners. For the most updated list of Mellanox trademarks, visit http://www.mellanox.com /page/trademarks Doc#: MLNX-15-50926 Mellanox Technologies 2

Table of Contents Table of Contents................................................ 3 Release Update History........................................... 4 Chapter 1 Overview.............................................. 5 Chapter 2 Changes and New Features History in IPsec Bundle............ 9 Chapter 3 Known Issues.......................................... 10 Chapter 4 Bug Fixes History....................................... 12 Mellanox Technologies 3

Release Update History Release Date Description 18.02.00 March 2018 Updated Section 1.1, Contents of Innova IPsec Bundle, on page 6 Updated Chapter 2, Changes and New Features History in IPsec Bundle on page 10 17.11.00 November 2017 Updated Section 1.1, Contents of Innova IPsec Bundle, on page 6 Updated Section 1.3, List of Supported Cables, on page 7 Updated Chapter 3, Known Issues on page 11 Updated Chapter 4, Bug Fixes History on page 13 17.08.00 August 2017 Updated Section 1.1, Contents of Innova IPsec Bundle, on page 6 Added Section 1.3, List of Supported Cables, on page 7 Updated Chapter 3, Known Issues on page 11 Updated Chapter 4, Bug Fixes History on page 13 17.06.00 June 2017 Updated Section 1.1, Contents of Innova IPsec Bundle, on page 6 Added Section 1.3, List of Supported Cables, on page 7 Updated Chapter 3, Known Issues on page 11 Updated Chapter 4, Bug Fixes History on page 13 17.01.00 January 31, 2017 Updated Section 1.1, Contents of Innova IPsec Bundle, on page 6 Updated Chapter 3, Known Issues on page 11 Updated Chapter 4, Bug Fixes History on page 13 December Bundle January 05, 2017 Updated Section 1.1, Contents of Innova IPsec Bundle, on page 6 Added Section 1.1.5, Scripts, on page 4 Updated Chapter 2, Changes and New Features History in IPsec Bundle on page 10 Updated Chapter 3, Known Issues on page 11 Updated Chapter 4, Bug Fixes History on page 13 Beta 10GbE and 40GbE September 29, 2016 Updated Section 1.1, Contents of Innova IPsec Bundle, on page 6 Updated Chapter 3, Known Issues on page 11 Added Chapter 4, Bug Fixes History on page 13 4 Mellanox Technologies

Release Date Description Alpha 10GbE and 40GbE Alpha 10GbE and 40GbE August 31, 2016 July 21, 2016 Removed Mellanox OFED Version First Release Mellanox Technologies 5

1 Overview These are the release notes for of Innova IPsec EN Adapter Card Bundle. Please refer to Section 1.1, Contents of Innova IPsec Bundle, on page 6 for the list of supported software, firmware and firmware tools versions. 1.1 Contents of Innova IPsec Bundle Content Description FPGA Image Version 3505-10GbE 3503-40GbE Firmware Version 14.22.1002 MFT Version a mft-4.9.0-38 - 10GbE/40GbE Driver/Kernel Components kernel-4.14.0_rc1_3bbaa83_2017_11_12_16-1.x86_64.rpm kernel-devel-4.14.0_rc1_3bbaa83_2017_11_12_16-1.x86_64.rpm iproute2-4.12.0rc4_ipsec_offload-1.x86_64.rpm Scripts xfrm-offload-transport.sh xfrm-offload-tunnel.sh MLNX_OFED 4.2 a. For MFT release notes, please refer to http://www.mellanox.com/page/management_tools. 1.2 Supported Platforms/Operating Systems Operating System RHEL 7.2 RHEL 7.3 RHEL7.4 x86_64 x86_64 X86_64 Platform 6 Mellanox Technologies

Overview 1.3 List of Supported Cables Table 1 - Validated and Supported 10/40GbE Cables Speed Cable OPN # Length Tested (m) Description MC2210130-001 1 Mellanox passive copper cable, ETH 40GbE,, QSFP, 1m MC2210130-002 2 Mellanox passive copper cable, ETH 40GbE,, QSFP, 2m MC2210128-003 3 Mellanox passive copper cable, ETH 40GbE,, QSFP, 3m MC2210126-004 4 Mellanox passive copper cable, ETH 40GbE,, QSFP, 4m MC2210126-005 5 Mellanox passive copper cable, ETH 40GbE,, QSFP, 5m MC2210411-SR4E (Module) MC2210411-SR4L (Module) MC2210411-SR4 (Module) NA NA NA Mellanox optical module,, QSFP, MPO, 850nm, up to 300m Mellanox optical module,, QSFP, MPO, 850nm, up to 30m MELLANOX OPTICAL MODULE 40GB/S QSFP MPO 850NM UP TO 100M FOR INTERNAL USE ONLY MC2210310-003 3 Mellanox active fiber cable, ETH 40GbE,, QSFP, 3m MC2210310-005 5 Mellanox active fiber cable, ETH 40GbE,, QSFP, 5m MC2210310-015 15 Mellanox active fiber cable, ETH 40GbE,, QSFP, 15m MC2210310-030 30 Mellanox active fiber cable, ETH 40GbE,, QSFP, 30m MC2210310-050 50 Mellanox active fiber cable, ETH 40GbE,, QSFP, 50m MC2210310-100 100 Mellanox active fiber cable, ETH 40GbE,, QSFP, 100m MC2210128-003 3 Mellanox passive copper cable, ETH 40GbE,, QSFP, 3m MC2210130-002 2 Mellanox passive copper cable, ETH 40GbE,, QSFP, 2m MC2210128-003 3 Mellanox passive copper cable, ETH 40GbE,, QSFP, 3m MC2210130-002 2 Mellanox passive copper cable, ETH 40GbE,, QSFP, 2m Mellanox Technologies 7

Table 1 - Validated and Supported 10/40GbE Cables Speed Cable OPN # Length Tested (m) Description MC2210128-003 3 Mellanox passive copper cable, ETH 40GbE,, QSFP, 3m MC2210130-002 2 Mellanox passive copper cable, ETH 40GbE,, QSFP, 2m QSFP-H40G-CU1M 1 Cisco QSFP-H40G-CU1M Compatible 40G QSFP+ Passive Direct Attach Copper Cable 1m QSFP-H40G-CU3M 3 Cisco QSFP-H40G-CU1M Compatible 40G QSFP+ Passive Direct Attach Copper Cable 3m QSFP-H40G-CU5M 5 Cisco QSFP-H40G-CU1M Compatible 40G QSFP+ Passive Direct Attach Copper Cable 5m QSFP-40G-SR-BD (AFBR-79EBPZ-CS2) NA 40G QSFP Bi-Directional transceiver module, Duplex Multi-mode Fiber, LC Duplex connector, 100m reach with OM3 fiber QSFP-40G-SR4 (AFBR-79EIPZ-CS1) NA 40GBASE-SR4 QSFP+ transceiver module for MMF, 4- lanes, 850-nm wavelength, 12-fiber MPO/MTP connector QSFP-40G-ACU10M (1111619001) 10 10m (33ft) Cisco QSFP-H40G-ACU10M Compatible 40G QSFP+ Active Direct Attach Copper Cable FCBN425QE1C10-C1 10 Finisar Quadwire 4x25Gb/s 100G Ethernet 10m QSFP- QSFP AOC MC2210130-002 2 Mellanox passive copper cable, ETH 40GbE,, QSFP, 2m MC2210128-003 3 Mellanox passive copper cable, ETH 40GbE,, QSFP, 3m 10Gb/s MC3309130-00A 0.5 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 0.5m 10Gb/s MC3309130-001 1 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 1m 10Gb/s MC3309130-002 2 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 2m 10Gb/s MC3309130-003 3 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 3m 10Gb/s MC3309124-004 4 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 4m 10Gb/s MC3309124-005 5 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 5m 10Gb/s MC3309124-006 6 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 6m 8 Mellanox Technologies

Overview Table 1 - Validated and Supported 10/40GbE Cables Speed Cable OPN # Length Tested (m) Description 10Gb/s MC3309124-007 7 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 7m 10Gb/s NDCCGF-C401 CISCO-AMPHENOL 1m 1 NDCCGF-0006 SFP28 TO SFP28, 1M, 30AWG 10Gb/s 10Gb/s 10Gb/s 74752-9096 CISCO- MOLEX INC 5m FTLX8571D3BCL- ME (Module MFM1T02A-SR) FTLX1471D3BCL (Module MFM1T02A- LR) 5 Dell Active DAC SFP+, Cisco PN SFP-H10GB-CU5M, Molex PN 74752-9096 NA NA Fiber Optic Transmitters, Receivers, Transceivers PIN 10GBASE-SR/SW 1200Mx-SNL-I trnscvr Finisar 10 GbE SR multimode SFP+ Transceiver, 10GBASE-SR, 850nm VCSEL, 0 C to 85 C, Duplex LC connector, 300m 10Gb/s MAM1Q00A-QSA-A NA Mellanox cable module, ETH 10GbE, to 10Gb/ s, SFP+ to QSFP 10Gb/s MC3309124-007 7 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 7m 10Gb/s MC3309130-002 2 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 2m 10Gb/s MC3309124-007 7 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 7m 10Gb/s MC3309130-002 2 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 2m 10Gb/s MC3309124-007 7 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 7m 10Gb/s MC3309130-002 2 Mellanox passive copper cable, ETH 10GbE, 10Gb/s, SFP+, 2m 10Gb/s SFP-H10GB-CU1M 1 SFP+ 10GBase-Cu SFP+ to SFP+ Direct Attach Cable Twinax Passive 1m / Cisco 10Gb/s SFP-H10GB-CU3M 3 SFP+ 10GBase-Cu SFP+ to SFP+ Direct Attach Cable Twinax Passive 3m / Cisco 10Gb/s SFP-H10GB-CU5M 5 SFP+ 10GBase-Cu SFP+ to SFP+ Direct Attach Cable Twinax Passive 5m / Cisco 10Gb/s SFP-10G-SR NA SFP+ 10GBase-SR MultiMode 850nm 300m DDM / Cisco Mellanox Technologies 9

2 Changes and New Features History in IPsec Bundle Table 2 - Changes and New Features History Feature/Change Description Collision Free Hashing Using Cuckoo hashing scheme resolves hash collisions. Rev 17.11.00 Extended Sequence Number The extended sequence number enables the use of 64-bit sequence numbers or extended (ESNs) for a particular ESP security association. Bug Fixes See Chapter 4, Bug Fixes History on page 13 Rev 17.08.00 Bug Fixes See Chapter 4, Bug Fixes History on page 13 Rev 17.06.00 Security protocols ESP over IPv6 Bug Fixes See Chapter 4, Bug Fixes History on page 13 Rev 17.01.00 Global Flow Control The IEEE 802.3x (Annex 31B of 802.3) flow control enables receiving buffer feedback (e.g. overflow) from a receiver to its sender. The pause action (XOFF) is a control frame sent by the receiver to alert the sender that the receiver buffer is stressed and will potentially be overflowed. The sender responds by stopping transmission of any new packets until the receiver is ready to accept them again. The pause frame contains a timeout value. The sender will wait during this timeout or until XON control message is received. Bug Fixes See Chapter 4, Bug Fixes History on page 13 December Bundle Mode Transport Mode Bug Fixes See Chapter 4, Bug Fixes History on page 13 Beta 10GbE and 40GbE Bug Fixes See Chapter 4, Bug Fixes History on page 13 Alpha 10GbE and 40GbE Security protocols ESP over IPv4 Mode Tunnel Mode Cryptographic algorithms AES-GCM 128/256-bit key, and 128-bit ICV 10 Mellanox Technologies

Known Issues 3 Known Issues The following table describes known issues in this bundle release and possible workarounds. For list of known issues for ConnectX-4 Lx firmware, please refer to http://www.mellanox.com/ page/firmware_table_connectx4lxen. Table 3 - Known Issues (Sheet 1 of 2) Internal Ref. - Description: This bundle is binary-incompatible with all previous bundles. Workaround: When upgrading to an updated bundle, the FPGA image must be updated first and all other components (FW, kernel, and tools) upgraded immediately after. Please following the below order when upgrading to an updated bundle: 1. FPGA image burn & load 2. FW burn and reset 3. Kernel installation 4. MFT installation Keywords: Bundle, binary-incompatible, FPGA image - Description: OFED installation on top of the kernel that is provided by this bundle is not supported. Keywords: OFED - Description: Packet Size (Actual Packet MTU) Limitation: The current offload implementation does not support IP fragmentation. The original packet size should be such that it does not exceed the interface s MTU size after the ESP transformation (encryption of the original IP packet which increases its length) and the headers (outer IP header) are added: Inner IP packet size <= I/F MTU - ESP additions (20) - outer_ip (20) - fragmentation issue reserved length (56) Inner IP packet size <= I/F MTU - 96 This mostly affects forwarded traffic into smaller MTU as well as UDP traffic. TCP does PMTU discovery by default and clamps the MSS accordingly. Keywords: MTU, limitation - Description: No LLC/SNAP support. Keywords: LLC/SNAP - Description: No support for FEC. When using switches, there may be a need to change its configuration. Keywords: FEC 955929 Description: Heavy traffic may cause SYN flooding. Keywords: SYN flooding Issue Mellanox Technologies 11

Table 3 - Known Issues (Sheet 2 of 2) Internal Ref. Issue - Description: Priority Based Flow Control is not supported Keywords: Priority Based Flow Control - Description: Pause configuration is not supported by driver. Default pause is global pause (enabled). Keywords: Global pause - Description: Supported MTU is between 512 and 2012 bytes. Setting MTU values outside of this range might fail or might cause the traffic loss. Workaround: Set MTU between 512 and 2012 bytes Keywords: MTU, driver 1146485 Description: After a link flap caused by the link partner, in rare cases physical errors appear on the Innova side. Workaround: Toggle the Innova link once. Keywords: Physical errors, link flap 1157770 Description: On Passthrough/VM machines with relatively old QEMU and libvirtd, CMD timeout might occur upon driver load. After timeout, no other commands will be completed and all driver operations will be stuck. Workaround: Upgrade the QEMU and libvirtd on the KVM server. Tested with the following versions: libvirt 2.1.0 QEMU 2.6.1 Keywords: MANAGE_PAGES, timeout 12 Mellanox Technologies

Bug Fixes History 4 Bug Fixes History The following table describes bug fixes in this bundle release. For list of bug fixes for ConnectX- 4 Lx firmware, please refer to http://www.mellanox.com/page/firmware_table_connectx- 4LxEN. Table 4 - Bug Fixes History (Sheet 1 of 3) Internal Ref. - Description: Configuring IPsec offload SA over VLAN is supported. Keywords: VLAN Discovered in Release: Beta 10GbE and 40GbE Fixed in Release: 17.08.00 Bundle 1003279 Description: Firmware asset no longer occurs after FPGA failsafe. Keywords: Firmware, failsafe Discovered in Release: 17.06.00 Bundle Fixed in Release: 17.08.00 Bundle 1050468 Description: Running load/reset command to FPGA no longer affects performing other operations. Keywords: FPGA, load/rest Discovered in Release: 17.06.00 Bundle Fixed in Release: 17.08.00 Bundle - Description: Driver no longer causes errors when attempting to configure incompatible offloaded XFRM. Keywords: Driver, XFRM Discovered in Release: Alpha 10GbE and 40GbE Fixed in Release: Beta 10GbE and 40GbE - Description: Added GSO support for offload. Keywords: GSO Discovered in Release: Alpha 10GbE and 40GbE Fixed in Release: Beta 10GbE and 40GbE - Description: FPGA load operation no longer causes the RDMA FPGA device to fail. Keywords: RDMA, load Discovered in Release: Alpha 10GbE and 40GbE Fixed in Release: December Bundle - Description: Added support of optical modules for 10GbE and 40GbE. Keywords: Optical modules Discovered in Release: Beta 10GbE and 40GbE Fixed in Release: December Bundle Issue Mellanox Technologies 13

Table 4 - Bug Fixes History (Sheet 2 of 3) Internal Ref. Issue - Description: The link no longer encounters stability issues after burning 10GbE image using mlx_fpga tool. Keywords: Link, 10GbE Discovered in Release: December Bundle Fixed in Release: 17.01.00 Bundle - Description: Image loading or resetting no longer results in FPGA error while loading messages. Keywords: Error messages Discovered in Release: Beta 10GbE and 40GbE Fixed in Release: December Bundle - Description: Green/Amber LED no longer blinks when cable is unplugged. Keywords: LED Discovered in Release: Beta 10GbE and 40GbE Fixed in Release: 17.01.00 Bundle - Description: Added support of LR4 modules for 10GbE and 40GbE. Keywords: LR4 Discovered in Release: December Bundle Fixed in Release: 17.01.00 Bundle - Description: Automatically loading esp4_offload is supported. Keywords: esp4_offload Discovered in Release: December Bundle Fixed in Release:17.01.00 Bundle - Description: The following errors no longer occur during FPGA reset: Error code 16 Error code 107 Internal link error Syndrome 0x828381 Keywords: FPGA reset, errors Discovered in Release: December Bundle Fixed in Release: 17.01.00 Bundle - Description: FPGA user temperature thresholds do not return to default after FPGA reset. Discovered in Release: December Bundle Fixed in Release: 17.06.00 Bundle 14 Mellanox Technologies

Bug Fixes History Table 4 - Bug Fixes History (Sheet 3 of 3) Internal Ref. Issue 959009 Description: After power cycle, loss of packets no longer occurs. Discovered in Release: December Bundle Fixed in Release: 17.06.00 Bundle - Description: UDP fragmentation is now supported over an IPsec SA if esp4_offload module is loaded. Discovered in Release: December Bundle Fixed in Release: 17.06.00 Bundle 913720 Description: Wrong port speed rate when running ethtool Discovered in Release: 17.01.00 Bundle Fixed in Release: 17.06.00 Bundle 869558 Description: Driver can be unloaded when there are offloaded security associations (SAs) configured. Keywords: SAs, mlx5_core driver Discovered in Release: 17.06.00 Bundle Fixed in Release: 17.11.00 Bundle 1045097 Description: Connecting and disconnecting a cable several times may cause a link up failure. Keywords: Cable, link up Discovered in Release: 17.06.00 Bundle Fixed in Release: 17.11.00 Bundle 928143 Description: DUP no longer occurs during firmware reset. Keywords: DUP Discovered in Release: 17.06.00 Bundle Fixed in Release: 17.11.00 Bundle 1044369 Description: FW indicates last loaded image after FPGA recovers from fatal temperature error. Keywords: FPGA, temperature Discovered in Release: 17.06.00 Bundle Fixed in Release: 17.11.00 Bundle Mellanox Technologies 15