Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes

Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes Part No: E74399-02 December 2016

Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes Part No: E74399-02 Copyright 2016, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS. Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government. This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle. Documentation Accessibility For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc. Access to Oracle Support Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup? ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Référence: E74399-02 Copyright 2016, Oracle et/ou ses affiliés. Tous droits réservés. Ce logiciel et la documentation qui l accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d utilisation et de divulgation. Sauf stipulation expresse de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, accorder de licence, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé que ce soit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d interopérabilité avec des logiciels tiers ou tel que prescrit par la loi. Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu elles soient exemptes d erreurs et vous invite, le cas échéant, à lui en faire part par écrit. Si ce logiciel, ou la documentation qui l accompagne, est livré sous licence au Gouvernement des Etats-Unis, ou à quiconque qui aurait souscrit la licence de ce logiciel pour le compte du Gouvernement des Etats-Unis, la notice suivante s applique: U.S. GOVERNMENT END USERS. Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government. Ce logiciel ou matériel a été développé pour un usage général dans le cadre d applications de gestion des informations. Ce logiciel ou matériel n est pas conçu ni n est destiné à être utilisé dans des applications à risque, notamment dans des applications pouvant causer des dommages corporels. Si vous utilisez ce logiciel ou matériel dans le cadre d applications dangereuses, il est de votre responsabilité de prendre toutes les mesures de secours, de sauvegarde, de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité. Oracle Corporation et ses affiliés déclinent toute responsabilité quant aux dommages causés par l utilisation de ce logiciel ou matériel pour ce type d applications. Oracle et Java sont des marques déposées d Oracle Corporation et/ou de ses affiliés. Tout autre nom mentionné peut correspondre à des marques appartenant à d autres propriétaires qu Oracle. Intel et Intel Xeon sont des marques ou des marques déposées d Intel Corporation. Toutes les marques SPARC sont utilisées sous licence et sont des marques ou des marques déposées de SPARC International, Inc. AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques déposées d Advanced Micro Devices. UNIX est une marque déposée d The Open Group. Ce logiciel ou matériel et la documentation qui l accompagne peuvent fournir des informations ou des liens donnant accès à des contenus, des produits et des services émanant de tiers. Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers, sauf mention contraire stipulée dans un contrat entre vous et Oracle. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l accès à des contenus, produits ou services tiers, ou à leur utilisation, sauf mention contraire stipulée dans un contrat entre vous et Oracle. Accessibilité de la documentation Pour plus d informations sur l engagement d Oracle pour l accessibilité à la documentation, visitez le site Web Oracle Accessibility Program, à l adresse http://www.oracle.com/ pls/topic/lookup?ctx=acc&id=docacc. Accès aux services de support Oracle Les clients Oracle qui ont souscrit un contrat de support ont accès au support électronique via My Oracle Support. Pour plus d informations, visitez le site http://www.oracle.com/ pls/topic/lookup?ctx=acc&id=info ou le site http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs si vous êtes malentendant.

Contents Late-Breaking Information... 7 Known Problem... 7 Hard Reset of CPUM Causes Non-Harmful Error Messages (24395799)... 7 General Information and Issues... 8 Minimum Supported Firmware Version... 8 Hardware Information and Issues... 8 10Gb Support on the Oracle F2 Quad Port 10GBASE-T Module... 9 Oracle F2 Long Range InfiniBand Module Deployment... 9 Software Information and Issues... 17 Oracle ILOM Issues... 17 Updating the Switch Firmware... 18 Documentation Information and Issues... 22 Oracle Fabric Interconnect F2-12 Documentation... 22 Connectivity Documentation... 22 Display Problem with Step Numbers in HTML Documentation... 23 Erroneous Product Name in Glossary for Some Oracle F2 I/O Module User's Guides... 23 5

6 Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Late-Breaking Information This document contains information about the Oracle Fabric Interconnect F2-12, (the virtualization switch) and Oracle F2 I/O modules. Additional release notes exist for the fabric management software. Refer to Oracle Fabric OS 1.0.2 Release Notes and Oracle Fabric Manager 5.0.2 Release Notes. This document contains the following topics: Known Problem on page 7 General Information and Issues on page 8 Hardware Information and Issues on page 8 Software Information and Issues on page 17 Documentation Information and Issues on page 22 Known Problem The following known problem exists in the switch hardware or management software, such as Oracle ILOM and SNMP. Hard Reset of CPUM Causes Non-Harmful Error Messages (24395799) on page 7 Hard Reset of CPUM Causes Non-Harmful Error Messages (24395799) On a virtualization switch running BIOS version 16, when you issue reset /SYS from ILOM CLI, the following events are logged in Oracle ILOM's event logs after the reboot. reset -script /SYS Performing hard reset on /SYS -> show -script /SP/logs/event/list Event Late-Breaking Information 7

General Information and Issues ID Date/Time Class ----- ------------------------ -------3 Sun Jan 9 04:07:55 2000 IPMI ID = a : 01/09/2000 : 04:07:55 : Bus Fatal Error : Asserted 2 Sun Jan 9 04:07:55 2000 IPMI ID = 9 : 01/09/2000 : 04:07:55 : Bus Fatal Error : Asserted 1 Sun Jan 9 04:07:40 2000 Reset Reset of /SYS by root succeeded.> Type -------Log Critical Severity -------minor Interrupt : SMI Handler : Log minor Critical Interrupt : SMI Handler : Log major These events are erroneous; no fatal error actually occurs. Avoidance: You can avoid encountering these events by using Oracle ILOM CLI command stop /SYSinstead of reset /SYS. Using stop /SYS causes a graceful shutdown of the switch, and prevents this error message from appearing in Oracle ILOM logs. Workaround: On a virtualization switch running BIOS revision 16, these messages are harmless and are safe to ignore if they occur immediately following the reset /SYS command. No action needs to be taken. General Information and Issues These topics document general information about the switches and I/O modules. Minimum Supported Firmware Version on page 8 Minimum Supported Firmware Version Depending on when your virtualization switch was delivered, the system firmware version can be either 1.0.1.0 or 1.0.2.0. Firmware version 1.0.2.0 is required, so if the virtualization switch is not running that version, you must update it. See Updating the Switch Firmware on page 18. Hardware Information and Issues These topics document additional information and issues about the Oracle Fabric Interconnect F2-12 and the Oracle F2 I/O modules. 8 10Gb Support on the Oracle F2 Quad Port 10GBASE-T Module on page 9 Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Hardware Information and Issues Oracle F2 Long Range InfiniBand Module Deployment on page 9 10Gb Support on the Oracle F2 Quad Port 10GBASE-T Module At the time of this release, the Oracle F2 Quad Port 10GBASE-T module supports only the 10Gbps speed. The 1Gbps and 100Mbps speeds are not supported. Oracle F2 Long Range InfiniBand Module Deployment The Oracle F2 Long Range InfiniBand Module (IB LR module) is an I/O module supported in the Oracle Fabric Interconnect F2-12. In documentation, the Oracle Fabric Interconnect F2-12 is referred to as the virtualization switch. The module is designed to support communication of InfiniBand traffic farther than the IBTA default distances. For more information about the IB LR module, see the Oracle F2 Long Range InfiniBand Module User s Guide. The IB LR Module supports a software feature called switch mode, which enables two virtualization switches on the same IB subnet to communicate over IB LR links at distances of up to a few hundred meters. This implementation enables connectivity for nearby, or adjacent, datacenters on the same campus for such uses as live mirroring or building-level disaster recovery within a campus. Switch-Mode Supported Topology on page 9 Switch-Mode Topology Considerations on page 11 Switch-Mode Configuration Considerations on page 11 Virtual Lane Independence Through IP Subnetting on page 12 Connect the Datacenters on page 15 Switch-Mode Supported Topology In switch mode, the topology is the overall deployment. The topology consists of sites, or datacenters, and each datacenter contains the virtualization switches that house the IB LR module(s). The diagram shows two distinct datacenters and how the hosts, Oracle InfiniBand IS2-46s (leaf switches), and virtualization switches should be connected. Notice that the IB LR links are the only connections that cross the datacenter boundary. For the connection procedure, see Connect the Datacenters on page 15. Late-Breaking Information 9

Hardware Information and Issues No. Description 1 Datacenter (shaded top area) consisting of hosts, leaf switches, and virtualization switches. 2 Datacenter (shaded bottom area) consisting of hosts, leaf switches, and virtualization switches. 3 Host connections 4x IB connections (magenta ports on the leaf switch). 4 Inter-switch connections, leaf and virtualization switches 12x IB connections (black ports on the leaf and virtualization switches). 5 Inter-switch connections, virtualization switches 4x IB connections (magenta and white connectors, ports 5 and 6) on each virtualization switch connect directly to the other virtualization switch in the same datacenter. 6, 7 10 Inter-datacenter connections, virtualization switches Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Hardware Information and Issues No. Description IB LR module links from a virtualization switch in one datacenter connect to each virtualization switch in the other datacenter. One single-mode fiber cable is required per connection. Switch-Mode Topology Considerations Switch mode's longer IB distances are supported only when specific guidelines are followed. As you read this topic, you will find it helpful to refer to the diagram in Switch-Mode Supported Topology on page 9. Topology and switch considerations: A maximum of two sites (datacenters) is supported per switch-mode topology. Within a switch-mode topology, a maximum of four virtualization switches is supported, two per datacenter so that each datacenter has redundant switches. Each virtualization switch must contain at least one IB LR module and the IB LR modules must be connected to the IB LR modules in the other virtualization switches. Module connectivity considerations: Link connectivity: Both sides of the IB LR links must be connected to the same outof-band (OOB) management network and be managed by a single admin authority that manages both sides of the IB LR link. Port connectivity: IB LR module ports can be connected to other IB LR module ports only. Generic QSFP-to-QSFP connectivity is not supported between two IB LR modules. Host server considerations: Switch mode is supported for stand-alone, rack-mount servers. Hosts must be connected to the Oracle InfiniBand IS2-46s (leaf switches). No other intervening IB device(s) can be used to connect the server HCA ports to the virtualization switch's external IB ports. Switch-Mode Configuration Considerations Before connecting the datacenters, be aware of the following: Switch mode uses the generic minhop routing algorithm to provide efficient, deadlockfree routing. To guarantee no deadlock conditions occur, the topology must ensure that the shortest paths between any communicating end nodes never imply that two or more flows all travel in the same direction in a ring of four or more connected switches. To prevent deadlock, ensure that different flows (for example, data traffic) that can contribute to deadlocks (for example, in combination with other data traffic) will use different Virtual Lanes (VLs). If you are using IPoIB in your deployment, you must define the VL independence by configuring specific IP subnets that map to IB service levels (SLs). See Virtual Lane Independence Through IP Subnetting on page 12. Late-Breaking Information 11

Hardware Information and Issues Any host-to-host data traffic that is not based on IPoIB must use the same SLs as documented in Virtual Lane Independence Through IP Subnetting on page 12. If the virtualization switches are used for standard server I/O as well as IB LR connectivity, the I/O traffic is limited to the local datacenter and cannot cross any IB LR link to the other datacenter. The I/O traffic must use the same SL or VL as is used for host-to-host communication within the local site. Oracle Fabric Manager can be used for OOB management. Oracle Fabric Manager requires specific ports be available to manage both sides of the LR links (for example, in case a firewall exists on the management network between the two datacenters). This table shows the ports OFM requires. Port Description 8443 (HTTPS) Used for configured users or admin user access to Oracle Fabric Manager. As an alternative, you can leave those ports closed, but open an RDP session and use the local browser for added security. 443 Used to configure Oracle Fabric Interconnects and Oracle SDN Controllers. 7443 Used to configure Oracle IB switches. 7777 Used to discover Oracle IB switches. 22 Used to download logs, and export and import backup files on all types of devices. In addition to the OFM requirements, the OOB coordination between subnet managers on each side of the IB LR link requires: Port 111 is available and not assigned to any other traffic. Port 111 is used for establishing RPC connections. Dynamically allocating port numbers for RPC connections must be allowed also. If you will use IB performance monitoring tools in the switch-mode topology, be aware that "global' gathering of performance management data is not supported. Instead, use these options: Performance data and other IB-counter information should be retrieved OOB through the leaf or virtualization switch's management interfaces. Standard IB tools can be used to collect counter information from one node at a time, but do not collect data from all nodes in a single operation. Only one statistics-gathering operation can be active at a time. Virtual Lane Independence Through IP Subnetting VL independence is enforced through IP networking by creating two separate IP subnets for each datacenter and one IP subnet for communication between the two datacenters. For example, assume datacenter 1 has subnets A and B, and datacenter 2 has subnets C and D, and subnet E is the communication subnet between the two datacenters. 12 Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Hardware Information and Issues Within each datacenter, hosts and virtualization switches have their own IP subnet, as shown in the following illustration. No. Description 1 Datacenter (shaded top area) consisting of hosts, leaf switches, and virtualization switches. 2 Datacenter (shaded bottom area) consisting of hosts, leaf switches, and virtualization switches. 3 IP Subnet, hosts (Subnet A) One IP subnet for the hosts maps to an IPoIB link in partition A and can be associated with SL2. The IP subnet for hosts is used for communication among the hosts in one datacenter. 4 IP Subnet, virtualization switches (Subnet B) Late-Breaking Information 13

Hardware Information and Issues No. Description One IP subnet for the virtualization switches maps to an IPoIB link in partition B and can be associated with SL3. The IP subnet for virtualization switches is used for communication between the pair of virtualization switches in one datacenter. If communication between virtualization switches is not required, these subnets can be left undefined. 5. IP Subnet, hosts (Subnet C) One IP subnet for the hosts in the second datacenter maps to an IPoIB link in partition C and can be associated with SL2. This subnet is conceptually the same as datacenter 1, but the IP subnet and partition key must be unique to the second datacenter. 6. IP Subnet, virtualization switches (Subnet D) One IP subnet for the virtualization switches in the second datacenter maps to an IPoIB link in partition D and can be associated with SL3. This subnet is conceptually the same as datacenter 1, but the IP subnet and partition key must be unique to the second datacenter. Between both datacenters, hosts in different datacenters can communicate over the single communication subnet as shown in the following illustration. 14 Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Connect the Datacenters No. Description 1 Datacenter (shaded top area) consisting of hosts, leaf switches, and virtualization switches. 2 Datacenter (shaded bottom area) consisting of hosts, leaf switches, and virtualization switches. 3 IP Subnet, common subnet for host communication between datacenters (Subnet E) One IP subnet maps to an IPoIB link in partition E associated with SL0. Connect the Datacenters To support switch mode's longer distances, the switches must be fully meshed, except for the leaf switches which should not directly connect to each other. As you perform this task, you will find it helpful to refer to the diagram in Switch-Mode Supported Topology on page 9. Late-Breaking Information 15

Connect the Datacenters To connect the datacenters, you connect the switches within both datacenters, then connect the datacenters together over the IB LR links, which are single-mode fiber cables. 1. Before beginning this procedure, read the configuration considerations: 2. 3. See Switch-Mode Topology Considerations on page 11 See Switch-Mode Configuration Considerations on page 11 In one datacenter, connect the virtualization switches together: a. Connect two magenta 4x IB cables to ports 5 and 6 on the virtualization switch. b. Connect the other end of the cables to the same ports on the other virtualization switch. Connect the leaf switches to the virtualization switches: a. Connect two black 12x IB cables to any two ports (1 through 4) on the leaf switch. b. Connect the other end of the cables to any 12x IB ports on the virtualization switch. c. Repeat these steps until each leaf switch is cross-connected to both virtualization switches. Caution - Do not directly connect the leaf switches to each other. 16 4. Connect each host to a 4x IB port on one leaf switch. 5. Connect each host to a 4x IB port on the other leaf switch. 6. In the other datacenter, repeat Step 2 through Step 5. 7. Connect the two datacenters: a. Connect one single-mode fiber cable to one of the ports on the IB LR module. b. Connect the other end of the cable to a port on an IB LR Module in one of the virtualization switches in the other datacenter. Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Software Information and Issues c. Connect a second single-mode fiber cable to one of the ports on the IB LR module. d. Connect the other end of the second cable to a port on an IB LR module in the other virtualization switch in the other datacenter. e. Repeat these steps to connect the IB LR links until each virtualization switch has two links connected, one link to each of the other virtualization switches in the other datacenter. Note - At this point, each virtualization switch should be connected to the other two virtualization switches in the other datacenter. 8. In each datacenter, connect to the respective management network. When connecting to datacenter management networks: Each leaf and virtualization switch must be connected to the datacenter management network in both data centers. It must be possible to communicate between the two data centers through the management network. Software Information and Issues These topics document software additional software information for the switches and I/O modules. Oracle ILOM Issues on page 17 Updating the Switch Firmware on page 18 Oracle ILOM Issues Though present in shipping switches, the Oracle ILOM target /SYS/COMe actually shows information for the CPUM chip in the switch. Also, there is no specific Oracle ILOM /SYS/ CPUM target. Individual Oracle F2 I/O modules are not supported in Oracle ILOM GUI. At this point, you cannot display or administer individual I/O modules through the Oracle ILOM GUI. Administration of Oracle F2 I/O modules is supported through the Oracle ILOM CLI, and consists of stopping, starting, resetting modules, and displaying module properties. In Oracle ILOM, fault management for I/O modules is not implemented yet, so error conditions for Oracle F2 I/O modules are not displayed in /System/Open_Problems. Late-Breaking Information 17

Software Information and Issues Updating the Switch Firmware These topics document how to update the switch firmware. Firmware Update Overview on page 18 Firmware Update Guidelines on page 18 Oracle Fabric OS Configuration Wizard on page 18 Acquire the Firmware Package on page 19 Update the Firmware on page 19 Verify the Update on page 21 Administer the Switch on page 21 Firmware Update Overview There are many components within the switch that use updatable firmware, for example the EMS and Oracle ILOM. The firmware package available from My Oracle Support includes firmware for each of these components. When you perform a firmware update, each of these components are updated as a function of the operation. It is not necessary to update each component separately. These are the main parts to the update process: Acquire the Firmware Package on page 19 Update the Firmware on page 19 Administer the Switch on page 21 Firmware Update Guidelines When updating the firmware, be aware of these considerations and guidelines: The Oracle ILOM load command is not supported for updating firmware from the SP. Multiple firmware packages exist on MOS. Make sure to get the correct version for your virtualization switch, which is patch number 24924488. The firmware is packaged in a zip file. Make sure you have sufficient free space to unzip the file. Upgrading the firmware causes the switch to power cycle, so schedule the firmware update during a service window. Oracle Fabric OS Configuration Wizard During the update process, be aware that the Oracle Fabric OS Configuration Wizard might start. This wizard is a tool that prompts you for decisions and information the switch needs to 18 Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Acquire the Firmware Package come online the first time it boots up or after significant changes to the configuration. Make sure to provide whatever information is required. Caution - You will need to specify the pool of MAC addresses available to the switch. MAC addresses must be unique for each virtualization switch. Using the same values on multiple virtualization switches will prevent high availability (HA) from operating and will require issuing the system clear-config command, restarting the Oracle Fabric OS Configuration Wizard, and then entering unique MAC addresses for each virtualization switch in the HA deployment. For more information, refer to the Oracle Fabric Interconnect F2-12 Installation Guide. Acquire the Firmware Package The virtualization switch's firmware packages are distributed through My Oracle Support (MOS). To retrieve firmware, you need to login with a valid MOS account. 1. Log in to MOS with a valid username and password. https://support.oracle.com 2. Locate and download firmware patch p24924488_1020_generic.zip. 3. If present, review the readme file while the firmware is downloading. 4. When the firmware is downloaded, decompress the archive or zip file if needed. 5. Start the update process. See Update the Firmware on page 19. Update the Firmware If you are upgrading from firmware version 1.0.1.0 to a later version, use this task to upgrade the firmware through the system upgrade command in Oracle Fabric OS. This command retrieves the firmware package as an.xpf file, loads it into switch memory, and upgrades appropriate system elements with relevant firmware. Also, during this process, the Oracle Fabric OS Configuration Wizard might run. See Oracle Fabric OS Configuration Wizard on page 18. Note - The firmware update process requires power cycling and rebooting the switch. This will affect the networks and fabric to which the switch is connected. Consider the consequences of these actions before proceeding. 1. Log in to the virtualization switch. Late-Breaking Information 19

Update the Firmware % ssh root@switch_host_ip_address root@switch_host_ip_address's password: password Last login: Fri Oct 14 16:20:27 2012 [root@switch_hostname ~]# 2. Access Oracle Fabric OS. [root@scp_hostname ~]# su admin Welcome to OFOS Controller Copyright (c) 2012-2016 Oracle Corp. All rights reserved. Enter "help" for information on available commands. Enter the command "show system copyright" for licensing information admin@scp_hostname[ofos] 3. Run the system upgrade command to update all relevant firmware on the virtualization switch. For example: system upgradefile_transfer_protocoluser_account@device_id:path_to_filename.xpf Where: file_transfer_protocol is the method of downloading the firmware package to the switch, either SCP, FTP, HTTP, or HTTPS. user_account is the name of an authorized user with appropriate privileges on the device that contains the firmware package. device_id is the IP address or host name of the device from which the firmware image can be downloaded. Note - You must include the colon ( : ) between device_id and path_and_filename. path_and_filename is the file path to the firmware package you downloaded, and the firmware package name including the.xpf suffix. For example: admin@scp_hostname[ofos] system upgrade scp://admin@192.168.112.126/oracle/updates/firmware/releases/ofos-1.0.251978.xpf admin@scp_hostname[ofos] 4. Answer any onscreen prompts to start the upgrade. While the firmware is upgrading, progress messages are displayed. 20 5. Allow the upgrade to run to completion. 6. Verify the upgrade occurred successfully. Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Verify the Update See Verify the Update on page 21. Verify the Update If you are updating from firmware version 1.0.1.0 to a later version, use this task to validate that the update has occurred successfully. 1. If you are not already in the Oracle Fabric OS, access it now. See Step 1 through Step 2. 2. Display the firmware versions. [admin@switch_hostname ~]# show system BASE OS: 2016-09-20_14-04-26 PSIF FW: 0.067 PSIF Bootloader: 0.036 SATURN: 1.0 BUILD-20160624-916 OFOS: 1.0.2-OFOS OFM: 5.0.2_OFM ILOM: 3.2.6.60-110640 EMS: 1.0.0 version completed with 0 error(s) [admin@switch_hostname ~]# 3. Verify the firmware update. [admin@switch_hostname ~]# system verify Verifying... ################################################# [100%] [admin@switch_hostname ~]# 4. (Optional) Configure the Oracle Fabric OS. See Administer the Switch on page 21. Administer the Switch After completing the Oracle Fabric OS Configuration Wizard (if needed), and verifying the update, you are now ready to configure your switch with connectivity and features appropriate for your datacenter, for example, VNICs, VLANS, and so on. 1. If you are not already in the Oracle Fabric OS, access it now. See Step 1 through Step 2. 2. Configure the Oracle Fabric OS. Refer to the Oracle Fabric OS 1.0.2 Administration Guide. Late-Breaking Information 21

Documentation Information and Issues Documentation Information and Issues These topics describe additional information or issues related to documentation. Oracle Fabric Interconnect F2-12 Documentation on page 22 Connectivity Documentation on page 22 Display Problem with Step Numbers in HTML Documentation on page 23 Erroneous Product Name in Glossary for Some Oracle F2 I/O Module User's Guides on page 23 Oracle Fabric Interconnect F2-12 Documentation The switch interacts with different products to provide end-to-end connectivity. The table shows the products directly related to the virtualization switch. Product Description Documentation Oracle Fabric Interconnect F2-12 (virtualization switch) Switch hardware and firmware Oracle Fabric Interconnect F2-12 Documentation Library Oracle F2 I/O modules I/O modules for customizing virtualization switch connectivity Oracle F2 I/O Modules Documentation Library Oracle Fabric OS 1.0.x Operating System and CLI for managing the switch Oracle Fabric OS 1.0.x Documentation Library Oracle Fabric Manager 5.0.x GUI for managing the switch Oracle Fabric Manager 5.0.x Documentation Library Oracle EDR InfiniBand Host Drivers Host drivers installed on hosts connected to the switch Oracle EDR InfiniBand Host Drivers Documentation Library Oracle EDR InfiniBand HCA Host Channel Adapter cards installed on hosts connected to the switch Oracle InfiniBand Dual Port HCA Documentation Library Documentation for each of the products is available through the Oracle Help Center: http://docs.oracle.com/en/networking/ In addition to this documentation, readme files are often available with the software or firmware that is downloadable from My Oracle Support (MOS). Make sure to read any readme file related to your downloaded files. Connectivity Documentation Also, a connectivity document exists which helps explain the cables and physical connections in the Oracle EDR switch family, including the Oracle F2 I/O modules supported by the virtualization switch. Refer to https://community.oracle.com/docs/doc-1006347. 22 Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016

Documentation Information and Issues Display Problem with Step Numbers in HTML Documentation In the HTML documentation only, some tasks have a problem that causes the displayed step number in a cross-reference to be incorrect. This problem is cosmetic only; it does not affect the functionality of the HTML documentation. When you click the HTML cross-reference, the correct target is displayed. As an alternative, you can use the PDF documentation. Erroneous Product Name in Glossary for Some Oracle F2 I/O Module User's Guides Document and Page Section Correction Oracle F2 Quad Port 10GBASE-T Module User's Guide, p. 36 Glossary The glossary entry for virtualization switch should be defined as Oracle Fabric Interconnect F2-12 Switch, not Oracle F2-Switch. Oracle F2 Quad Port 10GBASE-T Module User's Guide, p. 40 Oracle F2 Dual Port 16 Gb Fibre Channel Module User's Guide, p. 42 Late-Breaking Information 23

24 Oracle Fabric Interconnect F2-12 and Oracle F2 I/O Modules Product Notes December 2016