HP 6125XLG Blade Switch

Transcription

1 HP 6125XLG Blade Switch FCoE Configuration Guide Part number: Software version: Release 2306 Document version: 6W

2 Legal and notice information Copyright 2013 Hewlett-Packard Development Company, L.P. No part of this documentation may be reproduced or transmitted in any form or by any means without prior written consent of Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HEWLETT-PACKARD COMPANY MAKES NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIAL, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. Hewlett-Packard shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this material. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

3 Contents FCoE overview 1 Storage area network 1 FC SAN 1 FC protocol 2 Basic concepts 2 Communication flow 3 VSAN 4 FC zone 4 FCoE 5 Basic concepts 5 How FCoE works 7 FCoE modes 8 FCF mode 9 NPV mode 9 Protocols and standards 10 Configuring an FCoE mode 11 FCoE features supported in different FCoE modes 11 Configuration procedure 11 Configuring VFC interfaces and FIP 13 VFC interfaces and FIP configuration task list 13 Configuring a VFC interface 13 Enabling FCoE for a VLAN and mapping a VSAN to the VLAN 14 Configuration restrictions and guidelines 14 Configuration procedure 14 Configuring the FC-MAP value 15 Configuring the FKA advertisement period value 15 Configuring the FCF priority 16 Configuring the system FCF priority 16 Configuring the VFC interface FCF priority 16 Displaying and maintaining VFC interfaces and FIP 16 VFC interfaces and FIP configuration example 17 Network requirements 17 Configuration procedure 17 Setting up a fabric 21 Overview 21 Principal switch selection 21 Domain ID assignment 22 FC address assignment 23 Fabric setup configuration task list 23 Building a fabric statically 23 Building a fabric dynamically 24 Enabling or disabling the fabric configuration function 24 Setting a fabric name 25 Setting the switch priority 25 Configuring the allowed domain ID list 26 Configuring a domain ID for a switch 26 Configuring the mapping between the N_Port WWN and the FC address 27 i

4 Configuring the fabric timers 27 Configuring the fabric timers in system view 28 Configuring the fabric timers in VSAN view 28 Configuring the fabric reconfiguration 28 Configuring the auto fabric reconfiguration function 29 Manually initiating the fabric reconfiguration 29 Configuring a VFC interface to reject incoming RCF requests 29 Configuring and obtaining FC4 information of nodes 30 Enabling auto discovery of SCSI-FCP information 30 Configuring the default FC4 information for a node 31 Displaying and maintaining a fabric 31 Static fabric building configuration example 32 Network requirements 32 Configuration procedure 32 Verifying the configurations 36 Dynamic fabric building configuration example 37 Network requirements 37 Configuration procedure 38 Verifying the configurations 45 Configuring VSAN 47 VSAN fundamentals 47 Trunk VSAN in an FC network 47 Trunk VSAN in an FCoE network 47 Creating a VSAN 48 Configuring a trunk VSAN 48 Displaying and maintaining VSAN 48 VSAN configuration example 48 Network requirements 48 Configuration considerations 49 Configuration procedure 49 Verifying the configurations 52 Configuring FC routing and forwarding 54 Overview 54 Routing table and FIB table 54 Direct routes 55 Static routes 56 FSPF routes 56 Configuring static routes for FC 57 Configuration restrictions and guidelines 57 Configuration procedure 58 Configuring FSPF 58 FSPF configuration task list 58 Enabling FSPF 58 Configuring the shortest SPF calculation interval 59 Configuring the minimum LSR receiving interval 59 Configuring the minimum LSR refresh interval 59 Configuring the FSPF cost for an interface 60 Configuring the hello interval for an interface 60 Configuring the dead interval for an interface 60 Configuring the LSR retransmission interval for interfaces 61 Disabling FSPF for an interface 61 Configuring FSPF GR 62 Displaying and maintaining FC routing and forwarding 62 ii

5 Static FC routing configuration example 63 Network requirements 63 Configuration procedure 63 Verifying the configurations 66 FSPF configuration example 67 Network requirements 67 Configuration procedure 67 Verifying the configurations 69 Configuring FC zones 71 Overview 71 Zone database 71 Distributing zones 73 Zone merge 75 Access control 77 FC zone configuration task list 77 Configuring zone aliases 78 Configuring zones 78 Configuring zone sets 79 Configuring the default zone policy 79 Configuring zone distribution and merge types 79 Activating a zone set and distributing it to the entire fabric 80 Triggering a complete distribution 80 Renaming zone aliases, zones, and zone sets 81 Copying zone aliases, zones, and zone sets 81 Deleting the zone database 82 Displaying and maintaining FC zones 82 FC zone configuration example 82 Network requirements 82 Configuration considerations 83 Configuration procedure 83 Verifying the configurations 84 Configuring NPV 86 Overview 86 Downlink interface and downlink 86 Uplink interface and uplink 86 Downlink-to-uplink interface mappings 87 Disruptive load balancing 87 NPV configuration task list 87 Configuring uplink interfaces and downlink interfaces 87 Configuring uplink interfaces 88 Configuring downlink interfaces 88 Configuring downlink-to-uplink interface mappings 88 Initiating a disruptive load-balancing process 89 Displaying and maintaining NPV 89 NPV configuration example 89 Network requirements 89 Configuration procedure 90 Verifying the configurations 93 Configuring FC ping 95 Overview 95 Configuration procedure 95 FC ping configuration example 95 Network requirements 95 iii

6 Configuration procedure 96 Verifying the configurations 97 Configuring FC tracert 99 Overview 99 Configuration procedure 100 FC tracert configuration example 100 Network requirements 100 Configuration procedure 100 Appendixes 105 Appendix A Fabric address assignment 105 Appendix B Well-known fabric addresses 105 Support and other resources 107 Contacting HP 107 Subscription service 107 Related information 107 Documents 107 Websites 107 Conventions 108 Index 110 iv

7 FCoE overview The switch supports FCoE only when operating in advanced mode (the default). For more information about system operating modes, see Fundamentals Configuration Guide. HP recommends that you set the delay for the IRF ports to report a link down event as 0 on IRF member devices connected into a ring topology. For more information, see the irf link-delay command in Fundamentals Command Reference. Storage area network FC SAN According to the Storage Networking Industry Association dictionary, "a storage area network (SAN) is any high-performance network whose primary purpose is to enable disk devices to communicate with computer systems and with each other." A SAN enables the universal connectivity of servers and disk devices. Compared to the conventional client/server computer system, a SAN allows the servers to share data and directly access data created by one another without having to copy it, improves storage scalability, and centralizes the management of data backup, access, and security. Most SANs use Fibre Channel (FC) or Ethernet to interconnect devices. An FC SAN uses the FC protocol suite for communication, and an Ethernet SAN uses the TCP/IP protocol suite for communication. This document covers only the FC SAN. As shown in Figure 1, an FC SAN connects the data sending and receiving entities (network servers and disk devices) with fibers or copper wires in the following ways: Directly connects a server and a disk device, as shown in the point-to-point connection. Connects servers and disk devices to an FC switched fabric, as shown in the switched fabric. In a switched fabric, the servers and disk devices are called "nodes." A fabric uses 24-bit addressing and supports thousands of devices. 1

8 Figure 1 FC SAN networking (1) Point-to-point connection (2) Switched fabric Server Disk Fabric FC switch Server Disk NOTE: An FC SAN refers to a network comprising FC switches and nodes. A fabric refers to a transmission network comprising FC switches. FC protocol The servers, FC switches, and disk devices in an FC SAN must all support FC. Basic concepts WWN FC address The world wide name (WWN) is a 64-bit address that identifies a fabric or an entity (such as an FC switch, node, or port) in an FC SAN. The upper-layer protocol of FC uses WWNs for communication. Each entity has a factory-assigned globally unique WWN. The FC protocol accesses communication entities in a SAN through FC addresses. An FC address is also known as an "FC_ID." Figure 2 shows the structure of an FC address. The FC address is 24 bits long and is divided into these 8-bit fields: Domain_ID, Area_ID, and Port_ID. A domain represents an FC switch and all N_Ports connected to the switch. A Domain_ID, which is in the range of 1 to 239, uniquely identifies an FC switch. One or more N_Ports on the same node can be assigned to an area, which is identified by an Area_ID. The Port_ID field identifies an N_Port. 2

9 Figure 2 Structure of an FC address Port modes A Domain_ID can uniquely identify an FC switch. Different FC switches in the same fabric have different Domain_IDs. An FC address can uniquely identify an N_Port on a node. Different N_Ports on the same node have different FC addresses. FC switches use Domain_IDs to route messages between each other. The FC protocol standardizes the FC address usage. For more information, see "Appendixes." In a switched fabric, nodes and FC switches communicate through interfaces of different modes. Figure 3 Port modes N_Port N_Port Server Disk F_Port E_Port E_Port F_Port F_Port FC switch FC switch F_Port N_Port N_Port Disk Disk A node has the following port modes: N_Port Directly connects to a fabric. NL_Port Connects to a fabric through an arbitrated loop. An FC switch provides the following port modes: F_Port Connects to an N_Port or an NP_Port on another FC switch. E_Port Connects to an E_Port on another FC switch. NP_Port Connects to an F_Port on another FC switch. For more information about NP_Port, see "Configuring NPV." E_Ports connect FC switches to form a fabric, and F_Ports connect the nodes to FC switches in the fabric. Communication flow FC switches provide data transmission services. Through FC switches, a server sends instructions and data to disk devices and reads data from disk devices. 3

10 Figure 4 FC SAN communication model The following takes a server accessing a disk device as an example to see how data communication occurs in an FC SAN. 1. The server and the disk device use the fabric login (FLOGI) protocol to register with the FC switches, which then assign FC addresses to each directly-connected node. 2. The registered server and disk device send name service registration requests to their respective access FC switches to register name service information, including their WWNs and FC addresses. Finally, each FC switch in the fabric stores the name service information for all nodes. 3. To access a disk device, the server needs to send a name service query request to its directly-connected FC switch to obtain the list of disk devices in the fabric and their WWNs and FC addresses. 4. After the server obtains the FC address of the disk device, the server can send FC frames (with the FC address of the disk device as the destination FC address) to the FC switch nearby. 5. When the FC switch receives the FC frame from the server, it queries its FIB table for a data forwarding path according to the destination FC address, and forwards the FC frame to the next-hop FC switch. The next-hop FC switch forwards the FC frame in the same way, until the FC switch at the last hop forwards the FC frame to the destination disk device. NOTE: A FIB table is generated by the FC switch through calculation based on the FC routing protocol or configured static routes. VSAN FC zone In actual applications, the data is insecure because the data of all users is transmitted in the same FC SAN. You can divide one physical FC SAN into multiple Virtual Storage Area Networks (VSANs). In this manner, VSANs are separated from one another and provide independent services, enhancing adaptability and security of the network and offering more effective services for users. For more information about VSAN, see "Configuring VSAN." With VSAN, one physical SAN is divided into multiple logical SANs. A VSAN, however, cannot perform access control over the servers and disk devices (or the N_Ports) connected to a fabric. N_Ports in the same VSAN can access one another only if these N_Ports register name services. This creates data security risks. 4

11 FCoE Zoning can solve the preceding problem by dividing a VSAN into zones and adding N_Ports to different zones for different purposes. In this manner, N_Ports in different zones are separated to implement access control. For more information about FC zones, see "Configuring FC zones." A data center using the FC SAN technology usually comprises separate local area networks (LANs) and SANs. LANs carry traditional Ethernet/IP services, and SANs carry network storage services. To provide services for LANs and use SANs for storage simultaneously, the servers must use independent Ethernet adapters and FC adapters. In addition, the IP switches and the FC switches are also independent and have independent network connections. Such a network needs many switches, network adapters, and cables, and it brings high investments and maintenance costs and low scalability. FCoE was introduced to solve this problem. FCoE is a protocol that carries FC over Ethernet. In an FCoE solution, the server uses an FCoE-capable Ethernet adapter, and the FCoE switch (FCoE forwarder) integrates the functions of both the traditional IP switch and FCF switch. FCoE reduces the number of network adapters, switches, and cables, and the network operation and maintenance workload. In all, FCoE reduces the total cost. Figure 5 FCoE for I/O consolidation As shown in Figure 5, in the traditional network, the server is connected to the LAN through an Ethernet interface and to the SAN through an FC interface. In the FCoE network, the server is connected to the FCoE-capable FCF switch, and then the FCF switch is connected to the LAN through an Ethernet interface and to the SAN through an FC interface. The link between the server and the FCF switch can transmit both Ethernet frames and FC frames. Basic concepts As shown in Figure 6, the links between the FCF switch and the ENode (nodes that can transport FC over Ethernet, such as servers and disk devices) and between FCF switches can be used for receiving and sending both Ethernet frames and FC frames. 5

12 Figure 6 FCoE network diagram VFC interface and VN interface FIP protocol FCoE frames A virtual fiber channel (VFC) interface is a logical interface manually created on the FCF switch to simulate the function of a physical FC interface. To use a VFC interface, bind it to a physical Ethernet interface. You can connect either an ENode or an FCF switch to a VFC interface. VFC interfaces support E mode, F mode (default), and NP mode. The virtual node (VN) interface is a logical interface on an ENode to simulate the function of a physical FC interface. FCoE initialization protocol (FIP) is an FCoE control protocol that establishes and maintains virtual links. FIP establishes a virtual link between the VFC interface of an FCF switch and the VN interface of an ENode or between VFC interfaces of two FCF switches to provide a physical infrastructure for transmitting FC frames over Ethernet. To transmit an FC frame over an Ethernet link, you must encapsulate the FC frame in an FCoE frame by adding an Ethernet frame header to the FC frame. An FCoE frame uses Ethernet II encapsulation, which has the following fields in the Ethernet header: EtherType 0x8906. Destination MAC address/source MAC address For a switch, it is the FCoE MAC address of the switch (which can be displayed by using the display fcoe command). For a node, it is the fabric provided MAC address (FPMA) of the node. As shown in Figure 7, an FPMA is composed of the FC-MAP as the 24 most significant bits and the FC ID of the VN interface as the 24 least significant bits. The FC-MAP takes the value of the switch FC-MAP, 0x0EFC00 by default and configurable by using the fcoe fcmap command. 6

13 Figure 7 FPMA composition How FCoE works Figure 8 Block diagrams of the ENode and the FCF switch ENode FCF VN interface Virtual link VFC interface FC layer Ethernet layer FC layer Ethernet layer Ethernet interface Ethernet interface NOTE: This section describes how FCoE works only on the FCF switch, rather than on the ENode. Procedure for receiving and sending FC frames over Ethernet How FIP works An FC frame is transmitted over Ethernet using the following workflow: FIP establishes a virtual link between the VFC interface of the FCF switch and the VN interface of the ENode or between VFC interfaces of two FCF switches. After the virtual link is established, the FCF switch encapsulates the FC frame in an FCoE frame and sends it out. After receiving the FCoE frame, the FCF switch removes its Ethernet header to send the original FC frame to the upper layer for processing. FIP sets up and maintains virtual links between a VFC interface and a VN interface or between VFC interfaces. Two categories of packets are used in FIP: Discovery Solicitation and Discovery Advertisement. There are two types of Discovery Advertisement: Solicited Discovery Advertisement A reply for a Discovery Solicitation. Unsolicited Discovery Advertisement Periodically sent to advertise the presence of an FCF switch or maintain an existing virtual link. The following example shows how a virtual link is set up between an FCF switch and an ENode. 7

14 Figure 9 FIP operation FCF ENode (1) Send Discovery Solicitation Learn FCoE MAC address (2) Send solicited Discovery Advertisement (3) Send solicited Discovery Advertisements periodically (4) Send FLOGI request Check FCoE MAC address (5) Send FLOGI LS_ACC (6) Send solicited Discovery Advertisements periodically As shown in Figure 9, the following workflow is used to set up a virtual link: 1. The ENode sends a Discovery Solicitation containing its FCoE MAC address. 2. After receiving the Discovery Solicitation, the FCF switch acts differently depending on whether the receiving VFC interface is bound to the FCoE MAC address: If it is not bound, the switch learns the FCoE MAC address and replies with a solicited Discovery Advertisement, whose fcf priority field carries the FCF priority of the VFC interface. If it is bound, the switch checks whether the FCoE MAC address matches the bound FCoE MAC address. If they match, it replies with a solicited Discovery Advertisement, whose fcf priority field carries the FCF priority of the VFC interface. If they do not match, it discards the Discovery Solicitation. 3. The FCF switch periodically sends unsolicited Discovery Advertisements, whose fcf priority field carries the FCF priority of the system. The sending interval is specified by using the fcoe fka-adv-period command and defaults to 8 seconds. 4. After receiving the Discovery Advertisements, the ENode determines the FCF switch with the highest priority according to the fcf priority field and sends a FLOGI request frame to that switch for login. 5. After receiving the FLOGI request frame, the FCF checks whether the source MAC address matches its learned or bound FCoE MAC address. If they match, it sends a FLOGI LS_ACC, which indicates the setup of the virtual link. Otherwise, it discards the FLOGI frame. 6. The FCF switch also periodically sends unsolicited Discovery Advertisements to maintain established virtual links. If the ENode fails to receive an unsolicited Discovery Advertisement within a period 2.5 times the interval specified by the fcoe fka-adv-period command, it deletes the virtual link. FCoE modes The switch supports the following FCoE modes: 8

15 FCF mode FCF mode A switch operating in this mode is called an FCF switch. Its VFC interfaces support E mode (E_Port) and F mode (F_Port). NPV mode A switch operating in this mode is called an N_Port Virtualization (NPV) switch. Its VFC interfaces support F mode (F_Port) and NP mode (NP_Port). An FCoE-capable switch can operate in the following modes: FCF mode When the switch operates in this mode, it can connect to the E_Port on another FCF switch through its E_Port, or connect to the N_Port on a node or the NP_Port on an NPV switch through its F_Port. NPV mode When the switch operates in this mode, it can connect to the N_Port on a node through its F_Port or to the F_Port on an FCF switch through its NP_Port. Non-FCoE mode When the switch operates in this mode, it is a standard switch and does not provide any FCoE capabilities. An FCF switch encapsulates FC frames in Ethernet frames and uses FCoE virtual links to simulate physical FC links. Therefore, it provides standard FC switching capabilities and features on a lossless Ethernet network. Figure 10 FCF network diagram In an FCoE environment as shown in Figure 10, different from a pure FC network, the ENode and FCF switch communicate over Ethernet interfaces on a lossless Ethernet network. The FCoE virtual link between the ENode and FCF switch connects a VN interface to a VFC interface, and the FCoE virtual link between FCF switches connects two VFC interfaces. Each FCF switch is assigned a domain ID. Each FC SAN supports a maximum number of 239 domain IDs, so an FC SAN cannot have more than 239 FCF switches. NPV mode An FC SAN needs a large number of edge switches that connect directly to nodes. N_Port Virtualization (NPV) switches are developed to expand the number of switches in an FC SAN. 9

16 Figure 11 NPV network diagram As shown in Figure 11, the NPV switch resides between nodes and the core switch on the edge of the fabric. The core switch is a switch operating in FCF mode. The NPV switch is connected to the nodes through its F_Ports and to the core switch through its NP_Port. In this manner, the NPV switch forwards traffic from its connected nodes to the core switch. The NPV switch appears as an FCF switch to nodes and as a node to the core switch. For more information about NPV, see "Configuring NPV." Protocols and standards FC-FS-3, Fibre Channel - Framing and Signaling - 3 FC-SW-5, Fibre Channel - Switch Fabric - 5 FC-LS-2, Fibre Channel - Link Services - 2 FC-GS-6, Fibre Channel - Generic Services - 6 FC-BB-5, Fibre Channel - Back Bone 5 10

17 Configuring an FCoE mode The switch supports FCoE only when operating in advanced mode. For more information about system operating modes, see Fundamentals Configuration Guide. FCoE features supported in different FCoE modes The switch supports two FCoE modes: FCF mode and NPV mode. Each mode has different features as shown in Table 1. You can choose to configure different features based on the FCoE mode of a switch. Table 1 FCoE functions supported in different FCoE modes FCoE feature FCF mode NPV mode Configuring VFC interfaces and FIP Setting up a fabric Supported Supported Supported Only the following function is supported: "Configuring the fabric timers." Configuring VSAN Supported Supported Configuring FC routing and forwarding Supported Only the following functions are supported: Displaying FC routing table information Displaying FC FIB table information Display FC Exchange table information Configuring FC zones Supported Not supported Configuring NPV Not supported Supported Configuring FC ping Supported Not supported Configuring FC tracert Supported Not supported Configuration procedure An FCoE-capable switch can operate in FCF mode, NPV mode, or non-fcoe mode. The switch can only convert from non-fcoe mode to one FCoE mode, or vice versa, and it cannot convert directly among the two FCoE modes. To convert among the two FCoE modes, first convert the switch to non-fcoe mode. After converting the switch to non-fcoe mode, FCoE-related configurations in the original FCoE mode are cleared. To configure an FCoE mode for a switch: Step Command Remarks 1. Enter system view. system-view N/A 2. Configure an FCoE mode for the switch. fcoe-mode { fcf npv } By default, a switch operates in non-fcoe mode. 11

18 Step Command Remarks 3. Display the FCoE mode of the switch. display fcoe-mode This command can be executed in any view. 12

19 Configuring VFC interfaces and FIP VFC interfaces and FIP configuration task list Tasks at a glance (Required.) Configuring a VFC interface (Required.) Enabling FCoE for a VLAN and mapping a VSAN to the VLAN (Optional.) Configuring the FC-MAP value (Optional.) Configuring the FKA advertisement period value (Optional.) Configuring the FCF priority Configuring a VFC interface Step Command Remarks 1. Enter system view. system-view N/A 2. Create a VFC interface and enter its view. 3. Configure the VFC interface mode. 4. Bind the VFC interface to the specified Ethernet interface. 5. Assign the VFC interface to the specified VSAN as a trunk port. 6. (Optional.) Configure a description for the VFC interface. interface vfc interface-number fc mode { e f np } bind interface interface-type interface-number [ mac mac-address ] port trunk vsan vsan-id description text N/A By default, a VFC interface operates in F mode. When an FCF switch operates in FCF mode, VFC interfaces support E and F modes. When an FCF switch operates in NPV mode, FC interfaces support F and NP modes. By default, no Ethernet interface is bound to a VFC interface. The VFC interface sends and receives packets through the Ethernet interface bound to it. By default, a VFC interface is not assigned to any VSAN as a trunk port. You can assign a VFC interface to a nonexistent VSAN as a trunk port and then create the VSAN. By default, the description of an interface is Interface name Interface, for example, Vfc1 Interface. 13

20 Step Command Remarks 7. (Optional.) Restore the default settings for the VFC interface. default 8. Bring up the VFC interface. undo shutdown By default, a VFC interface is up. N/A Enabling FCoE for a VLAN and mapping a VSAN to the VLAN When you use a VFC interface to transmit packets, the Ethernet interface bound to the VFC interface can allow multiple VLANs. You should enable FCoE for a VLAN and map a VSAN to the VLAN, so that the packets from the VSAN are tagged with the VLAN tag and transmitted within the VLAN. Configuration restrictions and guidelines Follow these restrictions and guidelines when you configure this feature: FCoE cannot be enabled for VLAN 1. VSANs are mapped to VLANs on a one-to-one basis. You must enable FCoE for the same VLAN and map this VLAN to the same VSAN on the two ends. Make sure the Ethernet interface bound to the VFC interface allows the FCoE-capable VLAN. After you enable FCoE for a VLAN, the following changes apply to the VLAN: An FCoE-capable VLAN allows only FCoE traffic. All member ports in an FCoE-capable VLAN are isolated at Layer 2 to avoid loops. For this reason, STP and other loop detection protocols do not need to run in an FCoE-capable VLAN. Otherwise, FCoE links might be blocked. Because all member ports in an FCoE-capable VLAN are isolated, a Layer 2 protocol enabled in the FCoE-capable VLAN runs based on the port isolation topology. Configuration procedure To enable FCoE for the specified VLAN and map this VLAN to the specified VSAN: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VLAN view. vlan vlan-id N/A 3. Enable FCoE for the specified VLAN and map this VLAN to the specified VSAN. fcoe enable [ vsan vsan-id ] By default, FCoE for a VLAN is disabled. Make sure that the VSAN to be mapped has been created. 14

21 Configuring the FC-MAP value The FC-MAP value identifies an FCoE network. Switches in the same FCoE network must have the same FC-MAP value. IMPORTANT: After FC-MAP values are configured, VFC interfaces perform a renegotiation. The same FC-MAP value is required for two VFC interfaces to negotiate successfully. To configure an FC-MAP value: Step Command Remarks 1. Enter system view. system-view N/A 2. Configure an FC-MAP value. fcoe fcmap fc-map The default setting is 0x0EFC00. Configuring the FKA advertisement period value The FKA advertisement period determines the length of time it takes the switch to detect the disconnection of a virtual link. After setting up a virtual link with a peer switch, a switch sends unsolicited Discovery Advertisements every FKA advertisement period on its VFC interfaces in E mode to maintain the established virtual link. The FKA advertisement period value is carried in unsolicited Discovery Advertisements. After receiving an unsolicited Discovery Advertisement, the peer switch maintains the status of the virtual link and records the FKA advertisement period value. If the peer switch fails to receive an unsolicited Discovery Advertisement within 2.5 FKA advertisement periods, it deletes the virtual link. After setting up a virtual link with a peer ENode, the switch sends unsolicited Discovery Advertisements every FKA advertisement period on its VFC interfaces in F mode to maintain the established virtual link. The FKA advertisement period value is carried in unsolicited Discovery Advertisements. After receiving an unsolicited Discovery Advertisement, the peer ENode maintains the status of the virtual link and records the FKA advertisement period value. If the peer ENode fails to receive an unsolicited Discovery Advertisement within 2.5 FKA advertisement periods, it deletes the virtual link. In addition, the ENode sends keepalive frames to the switch every FKA advertisement period value (this value is obtained from unsolicited Discovery Advertisements received from the switch). After receiving a keepalive frame, the switch maintains the status of the virtual link. If the switch fails to receive a keepalive frame within 2.5 FKA advertisement periods, it deletes the virtual link. VFC interfaces in NP mode use the FKA advertisement period value learned from the peer switch instead of that configured on the local switch. According to FC-BB-5, the upper limit of the FKA advertisement period value is 90 seconds. Therefore, when the switch is connected to servers, storage devices, or third-party switches, configure the FKA advertisement period value to be no more than 90 seconds. To configure an FKA advertisement period value: 15

22 Step Command Remarks 1. Enter system view. system-view N/A 2. Configure an FKA advertisement period value. fcoe fka-adv-period fka-adv-period The default setting is 8 seconds. Configuring the FCF priority The FCF priority includes the VFC interface FCF priority and the system FCF priority, which are used in the following scenarios: The VFC interface FCF priority is used in the fcf priority field in an unsolicited Discovery Advertisement. The system FCF priority is used in the fcf priority field in a solicited Discovery Advertisement. An ENode selects the FCF switch with the highest priority from the FCF switches sending Discovery Advertisements and sends a FLOGI request to it for login. The FCF priority is effective only on a VFC interface connected to an ENode (VFC interface in F mode). Configuring the system FCF priority Step Command Remarks 1. Enter system view. system-view N/A 2. Configure the system FCF priority. fcoe global fcf-priority priority The default setting is 128. The configuration takes effect on all VFC interfaces in F mode. Configuring the VFC interface FCF priority Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. 3. Configure the FCF priority for the VFC interface. interface vfc interface-number fcoe fcf-priority priority N/A The default setting is 128. The configuration takes effect on a VFC interface only when it operates in F mode. Displaying and maintaining VFC interfaces and FIP Execute display commands in any view. 16

23 Task Display VFC interface information. Display FCoE global configuration. Command display interface [ vfc [ interface-number ] ] [ brief [ description ] ] display fcoe Clear the statistics for VFC interfaces. reset counters interface [ vfc [ number ] ] VFC interfaces and FIP configuration example Network requirements As shown in Figure 12, use the FCoE solution in a data center combining a LAN and a SAN to reduce the number of devices, network adapters, and cables. Figure 12 Network diagram If the FCF switch is connected to a server or storage device with a converged network adapter (CNA), you must also configure DCBX on the connecting Ethernet interface. For information about configuring DCBX, see Layer 2 LAN Switching Configuration Guide. Configuration procedure This section describes the configurations for VFC interfaces and FIP on the FCF switch. 1. Configure Switch A: # Configure Switch A to operate in advanced mode, save the configuration, and reboot Switch A. (Skip this step if the switch is operating in advanced mode.) <SwitchA> system-view [SwitchA] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode and create VSAN 10. <SwitchA> system-view 17

24 [SwitchA] fcoe-mode fcf [SwitchA] vsan 10 [SwitchA-vsan10] quit # Enable LLDP globally. [SwitchA] lldp global enable # Enable LLDP on interface Ten-GigabitEthernet 1/1/5, and enable the interface to advertise DCBX TLVs. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] lldp enable [SwitchA-Ten-GigabitEthernet1/1/5] lldp tlv-enable dot1-tlv dcbx [SwitchA-Ten-GigabitEthernet1/1/5] quit # Create an Ethernet frame header ACL numbered 4000, and configure two rules to match FCoE frames (protocol type 0x8906) and FIP frames (protocol type 0x8914). [SwitchA] acl number 4000 [SwitchA-acl-ethernetframe-4000] rule permit type 8906 ffff [SwitchA-acl-ethernetframe-4000] rule permit type 8914 ffff [SwitchA-acl-ethernetframe-4000] quit # Create a class named app_c with the logic OR operator, and specify ACL 4000 as the match criterion. [SwitchA] traffic classifier app_c operator or [SwitchA-classifier-app_c] if-match acl 4000 [SwitchA-classifier-app_c] quit # Create a behavior named app_b, and configure the action of marking packets with 802.1p priority 3. [SwitchA] traffic behavior app_b [SwitchA-behavior-app_b] remark dot1p 3 [SwitchA-behavior-app_b] quit # Create a QoS policy named plcy, and associate the class app_c with the behavior app_b in the QoS policy, specifying that the class-behavior association applies only to DCBX. [SwitchA] qos policy plcy [SwitchA-qospolicy-plcy] classifier app_c behavior app_b mode dcbx [SwitchA-qospolicy-plcy] quit # Apply the QoS policy plcy to the outbound direction of interface Ten-GigabitEthernet 1/1/5. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] qos apply policy plcy outbound [SwitchA-Ten-GigabitEthernet1/1/5] quit # Configure the mapping from 802.1p priority 3 to local precedence 3 in the outbound direction. (This is the default mapping, and you can modify the mapping as needed.) [SwitchA] qos map-table dot1p-lp [SwitchA-maptbl-out-dot1p-lp] import 3 export 3 [SwitchA-maptbl-out-dot1p-lp] quit # Enable byte-count WRR on interface Ten-GigabitEthernet 1/1/5, and configure queue 3 to use strict priority (SP) queuing. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] qos wrr byte-count [SwitchA-Ten-GigabitEthernet1/1/5] qos wrr 3 group sp 18

25 # Enable interface Ten-GigabitEthernet 1/1/5 to automatically negotiate with its peer to decide whether to enable PFC, enable PFC for 802.1p priority 3, and configure Ten-GigabitEthernet 1/1/5 to trust the 802.1p priority carried in packets. [SwitchA-Ten-GigabitEthernet1/1/5] priority-flow-control auto [SwitchA-Ten-GigabitEthernet1/1/5] priority-flow-control no-drop dot1p 3 [SwitchA-Ten-GigabitEthernet1/1/5] qos trust dot1p [SwitchA-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 1, bind it to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 10 as a trunk port. [SwitchA] interface vfc 1 [SwitchA-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchA-Vfc1] port trunk vsan 10 [SwitchA-Vfc1] quit # Create interface VFC 2, and configure it to operate in E mode. [SwitchA] interface vfc 2 [SwitchA-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 10 as a trunk port. [SwitchA-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchA-Vfc2] port trunk vsan 10 [SwitchA-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 20 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/5] port trunk permit vlan 20 [SwitchA-Ten-GigabitEthernet1/1/5] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 20 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/6] port trunk permit vlan 20 [SwitchA-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 20 and map VLAN 20 to VSAN 10. [SwitchA] vlan 20 [SwitchA-vlan20] fcoe enable vsan Configure Switch B: # Configure Switch B to operate in advanced mode, save the configuration, and reboot Switch B. (Skip this step if the switch is operating in advanced mode.) <SwitchB> system-view [SwitchB] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch B to operate in FCF mode, and create VSAN 10. <SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 10 [SwitchB-vsan10] quit 19

26 # Create interface VFC 2, and configure it to operate in E mode. [SwitchB] interface vfc 2 [SwitchB-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 10 as a trunk port. [SwitchB-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchB-Vfc2] port trunk vsan 10 [SwitchB-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 20 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/6 [SwitchB-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/6] port trunk permit vlan 20 [SwitchB-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 20 and map VLAN 20 to VSAN 10. [SwitchB] vlan 20 [SwitchB-vlan20] fcoe enable vsan 10 20

27 Setting up a fabric Overview A fabric transmits data for servers and disk devices. When setting up a fabric, you must assign a domain ID to each FCF switch in the fabric and assign an FC address to each node connected to the fabric. You can build a fabric through one of the following modes: Static mode You must manually assign domain IDs to all switches in the network, and then each switch assigns FC addresses to the N_Ports connected to it. The static mode avoids network flappings, but it is applicable only to simple, small-sized networks. Dynamic mode A principal switch is automatically elected to assign domain IDs to all switches in the network, and then each switch assigns FC addresses to the N_Ports connected to it. The dynamic mode enables centralized network management and is applicable to large-sized networks. Figure 13 Fabric setup workflows The following section details each process in the fabric setup workflows. Principal switch selection During the dynamic fabric building process, it is the principal switch that assigns domain IDs to all switches in the network. The switch with the highest priority is selected as the principal switch. When multiple switches have the same priority, the switch with the smallest WWN wins. The principal switch selection process is as follows: 1. When the principal switch selection starts, each switch considers itself as the principal switch, records itself in the principal switch information, and starts the Principal Switch Selection Timer (PSST), which is 10 seconds. 2. Before the PSST times out, the switches exchange packets carrying the principal switch information to select a principal switch. On receiving such a packet, a switch compares the priority and WWN of the principal switch carried in the packet against those locally recorded. If the priority carried 21

28 in the packet is higher, or the priority in the packet is the same and the WWN is smaller, the switch replaces the locally-record principal switch information with the principal switch information recorded in the packet, and notifies the other switches. Finally, all switches in the network make an agreement on which switch is the principal switch. 3. When the PSST times out, if the locally-recorded principal switch information is the local switch, the switch becomes the principal switch. After the principal switch is selected, the WWN of the principal switch becomes the fabric name. NOTE: During the principal switch selection process, if a switch receives a packet that updates the principal switch information, the switch must record the E_Port receiving the packet. The link relevant to this E_Port is called the "upstream principal link." Domain ID assignment A domain represents a switch and all N_Ports connected to the switch. Each domain must have a domain ID. An FCF switch can automatically assign domain IDs. Alternatively, you can manually configure static domain IDs. If you manually configure static domain IDs, you must assign a unique domain ID to each switch in the fabric. If domain IDs are dynamically assigned, the fabric configuration process is performed to select a principal switch and assign domain IDs. After the principal switch is selected, the principal switch assigns domain IDs to all switches in the fabric. After the fabric configuration process, each switch has a unique domain ID. The dynamic domain ID assignment process is as follows: 1. The principal switch assigns a domain ID to itself. If the principal switch has been configured with a desired domain ID, the principal switch assigns the domain ID to itself. Otherwise, the principal switch assigns the smallest available domain ID to itself. After the principal switch assigns a domain ID to itself, it notifies its downstream switches to request domain IDs from it. 2. After a downstream switch receives the notification, it starts to request a domain ID from the principal switch. If the downstream switch is configured with a desired domain ID, it requests the desired domain ID from the principal switch. 3. After the principal switch receives the domain ID request from the downstream switch, it assigns a domain ID to it and notifies the downstream switch. The principal switch assigns domain IDs following these rules: If the downstream switch requests a desired domain ID that is available, the principal switch assigns the domain ID to the downstream switch. If the downstream switch does not request a desired domain ID or the desired domain ID is not available, the principal switch assigns the smallest available domain ID to the downstream switch. If all available domain IDs have been assigned, the principal switch notifies the downstream switch that no domain ID can be assigned to it. 4. After the downstream switch receives the domain ID assignment notification from the principal switch, it works following these rules: 22

29 If the downstream switch has been configured with a static domain ID and the static domain ID is different from the one assigned by the principal switch, or if the principal switch notifies the downstream switch that no domain ID can be assigned, the downstream switch isolates its upstream principal link and brings down the relevant interface. For more information about domain ID types, see "Configuring a domain ID for a switch." Otherwise, the downstream accepts the domain ID assigned by the principal switch and notifies the nearby downstream switch to request a domain ID from the principal switch. 5. Repeat steps 2 through 4 until all downstream switches have been assigned domain IDs. NOTE: During the dynamic domain ID assignment process, if a switch receives the domain ID request on an E_Port, the switch must record the E_Port. The link relevant to this E_Port is called the "downstream principal link." FC address assignment After a switch obtains a domain ID, it can assign FC addresses to N_Ports directly connected. The Domain_ID field in the FC address is the domain ID of the switch, and it does not need assignment. The switch assigns the Area_IDs and Port_IDs following these guidelines: If you bind the WWN of an N_Port to an FC address, the switch assigns the bound FC address to the N_Port. If the N_Port itself has a desired FC address, the switch assigns the desired FC address, if available. If the N_Port itself does not have a desired FC address or the desired FC address is unavailable, the switch assigns the smallest available Area_ID and Port_ID to the N_Port. Fabric setup configuration task list When you set up a fabric, HP recommends that you use the same building mode (dynamic or static) for all switches in the fabric and then perform the following configurations depending on your building mode. Building a fabric statically Tasks at a glance (Required.) Configuring a VFC interface (Required.) Enabling or disabling the fabric configuration function (Required.) Setting a fabric name (Optional.) Configuring the allowed domain ID list Remarks N/A To statically build a fabric, you must disable the fabric configuration function. When statically building a fabric, you must manually configure the fabric name, and make sure all switches in the fabric are configured with the same fabric name. N/A 23

30 Tasks at a glance (Required.) Configuring a domain ID for a switch (Optional.) Configuring the mapping between the N_Port WWN and the FC address (Optional.) Configuring the fabric timers (Optional.) Configuring and obtaining FC4 information of nodes Remarks When statically building a fabric, you must manually configure a domain ID for each switch. N/A N/A N/A Building a fabric dynamically Tasks at a glance (Required.) Configuring a VFC interface (Required.) Enabling or disabling the fabric configuration function (Optional.) Setting the switch priority (Optional.) Configuring the allowed domain ID list (Optional.) Configuring a domain ID for a switch (Optional.) Configuring the mapping between the N_Port WWN and the FC address (Optional.) Configuring the fabric timers (Optional.) Configuring the fabric reconfiguration (Optional.) Configuring a VFC interface to reject incoming RCF requests (Optional.) Configuring and obtaining FC4 information of nodes Remarks N/A To dynamically build a fabric, you must enable the fabric configuration function. Principal switch selection relies on the switch priority. N/A When dynamically building a fabric, you can configure desired domain IDs for switches. N/A N/A N/A N/A N/A Enabling or disabling the fabric configuration function After being enabled with the fabric configuration function, FCF switches exchange messages to select the principal switch. Then the principal switch dynamically assigns domain IDs to all switches in the fabric. Therefore, to dynamically build a fabric, you must enable the fabric configuration function on switches. To statically set up a fabric, you must disable the fabric configuration function on switches and manually configure domain IDs for the switches. To enable or disable the fabric configuration function: Step Command Remarks 1. Enter system view. system-view N/A 24

31 Step Command Remarks 2. Enter VSAN view. vsan vsan-id N/A 3. Enable the fabric configuration function. 4. Disable the fabric configuration function. domain configure enable undo domain configure enable Enable or disable the function for all switches in the VSAN as required. By default, the fabric configuration function is enabled. Setting a fabric name The fabric name configured takes effect only on a statically-built fabric. You must configure the same fabric name for all switches in a VSAN. To set a fabric name: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure a fabric name. fabric-name name By default, the fabric name is null. If the user does not configure a fabric name, the switch WWN is used as the fabric name after FCoE is enabled. Setting the switch priority The priority value for FCF switches is in the range of 1 to 254. The smaller the value, the higher the priority. The FCF switch with the highest priority will be elected as the principal switch. The priority is configured on a per-vsan basis, and one FCF switch can have different priorities in different VSANs. In a stable fabric, the configured priority does not take effect immediately. Therefore, the running priority of a switch might be different from the configured priority. To validate the configured priority, use the domain restart disruptive command to perform a disruptive reconfiguration. After a disruptive reconfiguration, the running priority could still be different from the configured priority. See the following possibilities on the principal and a non-principal switch, depending on the configured priority value, as shown in Table 2. Table 2 Configured priority and running priority mappings Configured priority 2 > 2 Running priority Principal switch Same as the configured priority. Non-principal switch 3. Principal switch 2. Non-principal switch Same as the configured priority. 25

32 To set the switch priority: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure a priority value for the switch. priority value By default, the priority value of a switch is 128. Configuring the allowed domain ID list Configuring the allowed domain ID list has an effect on switches as follows: Principal switch Can only assign domains IDs within the allowed domain ID list. If the allowed domain ID list configured does not include any of the already assigned domain IDs or manually configured domain IDs, the configuration will fail. Non-principal switch The manually configured domain ID must be within the allowed domain ID list. Otherwise, the configuration will fail. The principal switch must assign the switch a domain ID within the allowed domain ID list. Otherwise, the switch refuses the assigned domain ID and isolates its interface connected to the principal switch. If the runtime domain ID for a switch is beyond the new allowed ID list, the configuration will also fail. HP recommends that you specify the same allowed domain ID list for the member switches of a VSAN. To configure the allowed domain IDs for a switch: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure the allowed domain IDs for the switch. allowed-domain-id domain-id-list By default, the allowed domain IDs are 1 to 239. Configuring a domain ID for a switch In different scenarios, the configured domain ID has different meanings. In a statically built fabric, the configured domain ID is the actual domain ID of the switch. In a dynamically built fabric, the configured domain ID is desired by the switch but might not be the actual domain ID. To statically build a fabric, you must manually configure a domain ID for each switch. To dynamically build a fabric, you can configure a desired domain ID for a switch, but the domain ID assigned to the switch might not be the desired one. The configured domain ID can be static type or preferred type. In a statically built fabric, the two types make no difference. In a dynamically built fabric, when a non-principal switch fails to obtain the desired domain ID from the principal switch, the non-principal switch can use another domain ID assigned by the principal 26

33 switch if the preferred type is configured. The non-principal switch will isolate the upstream link and refuse other domain IDs assigned by the principal switch if the static type is configured. HP recommends that you configure domain IDs of the same type for all switches in a VSAN. To configure a domain ID for a switch: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure a domain ID for the switch. domain-id domain-id { preferred static } By default, the domain ID of a switch is 0 and is of the preferred type. Configuring the mapping between the N_Port WWN and the FC address If you bind the WWN of an N_Port to an FC address, when the N_Port requests an FC address, the switch assigns the bound FC address to it. The WWN of an N_Port can be bound to only one FC address, and vice versa. The N-Port here indicates an N_Port on a node or an NP_Port on an NPV switch. To bind the WWN of an N_Port to an FC address: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Bind the WWN of an N_Port to an FC address. wwn wwn-value area-port-id area-port-id-value By default, no binding is configured. If the N_Port has been assigned another FC address or the FC address has been assigned to another N_Port, the binding is not allowed. Configuring the fabric timers The fabric operation involves the following timers: Distributed service timeout period Error detection timeout period Resource allocation timeout period For more information about these timers, see FC-related protocols and standards. You can configure fabric timers by using one of the following ways: Configure the timers in system view, and the configuration takes effect for all VSANs. 27

34 Configure the timers in VSAN view, and the configuration takes effect for only the VSAN. If you perform the configuration in both system view and VSAN view, the configuration made in VSAN view applies to the VSAN. Configuring the fabric timers in system view Step Command Remarks 1. Enter system view. system-view N/A 2. Configure the global distributed service timeout period. 3. Configure the global error detection timeout period. 4. Configure the global resource allocation timeout period. fc timer distributed-services value fc timer error-detect value fc timer resource-allocation value By default, the distributed service timeout period is 5000 milliseconds. By default, the error detection timeout period is 2000 milliseconds. By default, the resource allocation timeout period is milliseconds. Configuring the fabric timers in VSAN view Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure the distributed service timeout period for the VSAN. 4. Configure the error detection timeout period for the VSAN. 5. Configure the resource allocation timeout period for the VSAN. timer distributed-services value timer error-detect value timer resource-allocation value By default, the distributed service timeout period is 5000 milliseconds. By default, the error detection timeout period is 2000 milliseconds. By default, the resource allocation timeout period is milliseconds. Configuring the fabric reconfiguration IMPORTANT: The fabric reconfiguration function takes effects only when the fabric configuration function is enabled. The fabric reconfiguration occurs in the case of a network reconstruction (for example, two fabrics are merged) or external intervention (for example, the administrator uses a command to initiate reconfiguration). The fabric reconfiguration triggers principal switch selection, domain ID assignment, and FC address assignment throughout the fabric. The fabric reconfiguration includes the following categories: 28

35 Disruptive reconfiguration Floods the Reconfigure Fabric (RCF) frames throughout the fabric, and notifies all switches to perform a disruptive reconfiguration. During the reconfiguration procedure, each switch clears all data for renegotiation, and data transmission in the fabric is disrupted. Non-disruptive reconfiguration Floods the Build Fabric (BF) frames throughout the fabric, and notifies all switches to perform a non-disruptive reconfiguration. During the reconfiguration procedure, each switch tries to save the last running data for its domain ID to remain unchanged. Thus, data transmission in the fabric is not disrupted. Depending on the triggering conditions, the fabric reconfiguration includes auto reconfiguration and manual reconfiguration. When two fabrics are merged: The switches automatically perform a disruptive reconfiguration if the domain ID lists overlap. The system automatically performs a non-disruptive reconfiguration if the principal switch information of the two fabrics is different and the domain ID lists are not empty or overlapping. You can manually initiate a disruptive reconfiguration to trigger the fabric reconfiguration if ports are isolated and priority values of switches are modified. When the principal switch in a fabric is down, the system automatically performs a non-disruptive reconfiguration. Configuring the auto fabric reconfiguration function Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Enable the auto fabric reconfiguration function. domain auto-reconfigure enable By default, the auto fabric reconfiguration function is disabled. Manually initiating the fabric reconfiguration Step Command 1. Enter system view. system-view 2. Enter VSAN view. vsan vsan-id 3. Initiate the fabric reconfiguration. domain restart [ disruptive ] Configuring a VFC interface to reject incoming RCF requests In a stable fabric, to avoid unnecessary disruptive reconfigurations, you can configure a VFC interface to reject the RCF requests received from a specific VSAN by replying with a reject message. With this feature, the interface on which an RCF request is received is isolated. To configure a VFC interface to reject the received RCF requests: 29

36 Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. interface vfc interface-number N/A 3. Configure the VFC interface to reject the received RCF requests. fc domain rcf-reject vsan vsan-id By default, a VFC interface does not reject the received RCF requests. Configuring and obtaining FC4 information of nodes After a node registers with a switch through a FLOGI, the node sends a name service registration request to the switch for registering extended information, which includes FC4 information. FC4 information describes the FC4-layer protocol supported by a node and the feature of the supported protocol by using the following fields: FC4-Type FC4-layer protocol supported by a node: IP. NPV. SCSI-FCP. SNMP. Feature Feature of the supported FC4-layer protocol. Each FC4-layer protocol can have one of the four features and defines each feature. Before communication, a node obtains information about all nodes that support SCSI-FCP from the switch. You can use the display fc name-service database command to display the FC4 information of nodes in the name service database. Typically, when registering extended information, a server registers SCSI-FCP for FC4-Type and Initiator for Feature; a disk device registers SCSI-FCP for FC4-Type and Target for Feature. As a result, servers can determine the disk devices to send access requests after obtaining information about nodes supporting SCSI-FCP. Enabling auto discovery of SCSI-FCP information In some cases, for example, when a node logs out and then logs in, the node does not register SCSI-FCP support and therefore does not have a Feature value. This might cause the communication failure between the node and other nodes. With auto discovery of SCSI-FCP information, when a node logs in, the switch automatically obtains the SCSI-FCP support and the Feature value by sending a PRLI packet to the node. Then, the switch stores the SCSI-FCP information in the name service database. When auto discovery of SCSI-FCP information is enabled, some old network adapters might stop automatically registering the node information with the switch. Enable auto discovery of SCSI-FCP information as needed. To enable auto discovery of SCSI-FCP information: 30

37 Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Enable auto discovery of SCSI-FCP information. fc name-service auto-discovery By default, auto discovery of SCSI-FCP information is enabled. Configuring the default FC4 information for a node If a node does not register FC4 information and the switch fails to obtain SCSI-FCP information from the node, the switch records the default FC4 information in the name service database for that node. When a node registers FC4 information or the switch obtains the SCSI-FCP information, the switch replaces the default FC4 information with the registered FC4 information or obtained SCSI-FCP information. The fc wwn default-fc4-type command can be used to configure only one combination of FC4-Type and Feature at a time. To configure the default FC4 information for a node: Step Command Remarks 1. Enter system view. system-view N/A 2. Configure the default FC4 information for a node. fc wwn wwn-value default-fc4-type { type-value feature feature-map scsi-fcp feature { feature-map both initiator target } } By default, no default FC4 information is configured. Displaying and maintaining a fabric Execute display commands in any view. Task Display the domain information of the specified VSAN. Display the list of domain IDs dynamically allocated in the specified VSAN. Display fabric timers. Command display fc domain [ vsan vsan-id ] display fc domain-list [ vsan vsan-id ] display fc timer [ distributed-services error-detect resource allocation ] [ vsan vsan-id ] Display node login information. display fc login [ vsan vsan-id ] [ count ] Display the SCR table for an N_Port. display fc scr-table [ vsan vsan-id ] Display name service database information. display fc name-service database [ vsan vsan-id [ fcid fcid ] ] [ verbose ] Display ESS negotiation results. display fc ess [ vsan vsan-id ] 31

38 Static fabric building configuration example Network requirements As shown in Figure 14, use the static method to build a fabric. Figure 14 Network diagram Configuration procedure 1. Configure Switch A: # Configure Switch A to operate in advanced mode, save the configuration, and reboot Switch A. (Skip this step if the switch is operating in advanced mode.) <SwitchA> system-view [SwitchA] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode, and disable the fabric configuration function in VSAN 1. <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 1 [SwitchA-vsan1] undo domain configure enable # Configure a fabric name. [SwitchA-vsan1] fabric-name 11:11:11:11:11:11:11:11 # Configure the domain ID as 1. [SwitchA-vsan1] domain-id 1 static Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchA-vsan1] quit # Enable LLDP globally. [SwitchA] lldp global enable # Enable LLDP on interface Ten-GigabitEthernet 1/1/5, and enable the interface to advertise DCBX TLVs. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] lldp enable [SwitchA-Ten-GigabitEthernet1/1/5] lldp tlv-enable dot1-tlv dcbx [SwitchA-Ten-GigabitEthernet1/1/5] quit # Create an Ethernet frame header ACL numbered 4000, and configure two rules to match FCoE frames (protocol type 0x8906) and FIP frames (protocol type 0x8914). 32

39 [SwitchA] acl number 4000 [SwitchA-acl-ethernetframe-4000] rule permit type 8906 ffff [SwitchA-acl-ethernetframe-4000] rule permit type 8914 ffff [SwitchA-acl-ethernetframe-4000] quit # Create a class named app_c with the logic OR operator, and specify ACL 4000 as the match criterion. [SwitchA] traffic classifier app_c operator or [SwitchA-classifier-app_c] if-match acl 4000 [SwitchA-classifier-app_c] quit # Create a behavior named app_b, and configure the action of marking packets with 802.1p priority 3. [SwitchA] traffic behavior app_b [SwitchA-behavior-app_b] remark dot1p 3 [SwitchA-behavior-app_b] quit # Create a QoS policy named plcy, and associate the class app_c with the behavior app_b in the QoS policy, specifying that the class-behavior association applies only to DCBX. [SwitchA] qos policy plcy [SwitchA-qospolicy-plcy] classifier app_c behavior app_b mode dcbx [SwitchA-qospolicy-plcy] quit # Apply the QoS policy plcy to the outbound direction of interface Ten-GigabitEthernet 1/1/5. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] qos apply policy plcy outbound [SwitchA-Ten-GigabitEthernet1/1/5] quit # Configure the mapping from 802.1p priority 3 to local precedence 3 in the outbound direction. (This is the default, and you can modify the mapping as needed.) [SwitchA] qos map-table dot1p-lp [SwitchA-maptbl-out-dot1p-lp] import 3 export 3 [SwitchA-maptbl-out-dot1p-lp] quit # Enable byte-count WRR on interface Ten-GigabitEthernet 1/1/5, and configure queue 3 to use SP queuing. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] qos wrr byte-count [SwitchA-Ten-GigabitEthernet1/1/5] qos wrr 3 group sp # Enable interface Ten-GigabitEthernet 1/1/5 to automatically negotiate with its peer to decide whether to enable PFC, enable PFC for 802.1p priority 3, and configure Ten-GigabitEthernet 1/1/5 to trust the 802.1p priority carried in packets. [SwitchA-Ten-GigabitEthernet1/1/5] priority-flow-control auto [SwitchA-Ten-GigabitEthernet1/1/5] priority-flow-control no-drop dot1p 3 [SwitchA-Ten-GigabitEthernet1/1/5] qos trust dot1p [SwitchA-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 1, configure it to operate in F mode, bind it to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. [SwitchA] interface vfc 1 [SwitchA-Vfc1] fc mode f [SwitchA-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchA-Vfc1] port trunk vsan 1 [SwitchA-Vfc1] quit 33

40 # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchA-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 2, and configure it to operate in E mode. [SwitchA] interface vfc 2 [SwitchA-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchA-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchA-Vfc2] port trunk vsan 1 [SwitchA-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchA-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 1. [SwitchA] vlan 10 [SwitchA-vlan10] fcoe enable vsan 1 [SwitchA-vlan10] quit 2. Configure Switch B: # Configure Switch B to operate in advanced mode, save the configuration, and reboot Switch B. (Skip this step if the switch is operating in advanced mode.) <SwitchB> system-view [SwitchB] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch B to operate in FCF mode, and disable the fabric configuration function in VSAN 1. <SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 1 [SwitchB-vsan1] undo domain configure enable # Configure a fabric name. [SwitchB-vsan1] fabric-name 11:11:11:11:11:11:11:11 # Configure the domain ID as 2. [SwitchB-vsan1] domain-id 2 static Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchB-vsan1] quit # Enable LLDP globally. [SwitchB] lldp global enable 34

41 # Enable LLDP on interface Ten-GigabitEthernet 1/1/5, and enable the interface to advertise DCBX TLVs. [SwitchB] interface ten-gigabitethernet 1/1/5 [SwitchB-Ten-GigabitEthernet1/1/5] lldp enable [SwitchB-Ten-GigabitEthernet1/1/5] lldp tlv-enable dot1-tlv dcbx [SwitchB-Ten-GigabitEthernet1/1/5] quit # Create an Ethernet frame header ACL numbered 4000, and configure two rules to match FCoE frames (protocol type 0x8906) and FIP frames (protocol type 0x8914). [SwitchB] acl number 4000 [SwitchB-acl-ethernetframe-4000] rule permit type 8906 ffff [SwitchB-acl-ethernetframe-4000] rule permit type 8914 ffff [SwitchB-acl-ethernetframe-4000] quit # Create a class named app_c with the logic OR operator, and specify ACL 4000 as the match criterion. [SwitchB] traffic classifier app_c operator or [SwitchB-classifier-app_c] if-match acl 4000 [SwitchB-classifier-app_c] quit # Create a behavior named app_b, and configure the action of marking packets with 802.1p priority 3. [SwitchB] traffic behavior app_b [SwitchB-behavior-app_b] remark dot1p 3 [SwitchB-behavior-app_b] quit # Create a QoS policy named plcy, and associate the class app_c with the behavior app_b in the QoS policy, specifying that the class-behavior association applies only to DCBX. [SwitchB] qos policy plcy [SwitchB-qospolicy-plcy] classifier app_c behavior app_b mode dcbx [SwitchB-qospolicy-plcy] quit # Apply the QoS policy plcy to the outbound direction of interface Ten-GigabitEthernet 1/1/5. [SwitchB] interface ten-gigabitethernet 1/1/5 [SwitchB-Ten-GigabitEthernet1/1/5] qos apply policy plcy outbound [SwitchB-Ten-GigabitEthernet1/1/5] quit # Configure the mapping from 802.1p priority 3 to local precedence 3 in the outbound direction. (This is the default, and you can modify the mapping as needed.) [SwitchB] qos map-table dot1p-lp [SwitchB-maptbl-out-dot1p-lp] import 3 export 3 [SwitchB-maptbl-out-dot1p-lp] quit # Enable byte-count WRR on interface Ten-GigabitEthernet 1/1/5, and configure queue 3 to use SP queuing. [SwitchB] interface ten-gigabitethernet 1/1/5 [SwitchB-Ten-GigabitEthernet1/1/5] qos wrr byte-count [SwitchB-Ten-GigabitEthernet1/1/5] qos wrr 3 group sp # Enable interface Ten-GigabitEthernet 1/1/5 to automatically negotiate with its peer to decide whether to enable PFC, enable PFC for 802.1p priority 3, and configure Ten-GigabitEthernet 1/1/5 to trust the 802.1p priority carried in packets. [SwitchB-Ten-GigabitEthernet1/1/5] priority-flow-control auto [SwitchB-Ten-GigabitEthernet1/1/5] priority-flow-control no-drop dot1p 3 [SwitchB-Ten-GigabitEthernet1/1/5] qos trust dot1p 35

42 [SwitchB-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 1, configure it to operate in F mode, bind it to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. [SwitchB] interface vfc 1 [SwitchB-Vfc1] fc mode f [SwitchB-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchB-Vfc1] port trunk vsan 1 [SwitchB-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/5 [SwitchB-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 2, and configure it to operate in E mode. [SwitchB] interface vfc 2 [SwitchB-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchB-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchB-Vfc2] port trunk vsan 1 [SwitchB-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/6 [SwitchB-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 1. [SwitchB] vlan 10 [SwitchB-vlan10] fcoe enable vsan 1 [SwitchB-vlan10] quit Verifying the configurations 1. Verify the configurations on Switch A. [SwitchA] display fc domain vsan 1 Domain Information of VSAN 1: Running time information: State: Stable Switch WWN: 48:33:43:2d:46:43:1A:1A Fabric name: 11:11:11:11:11:11:11:11 Priority: 128 Domain ID: 1 Configuration information: Domain configure: Disabled Domain auto-reconfigure: Disabled Fabric name: 11:11:11:11:11:11:11:11 36

43 Priority: 128 Domain ID: 1 (static) Principal switch running time information: Priority: 128 No interfaces available. The output shows that the domain configuration is complete and that the runtime domain ID of Switch A is Verify the configurations on Switch B. [SwitchB] display fc domain vsan 1 Domain Information of VSAN 1: Running time information: State: Stable Switch WWN: 48:33:43:2d:46:43:1B:1B Fabric name: 11:11:11:11:11:11:11:11 Priority: 128 Domain ID: 2 Configuration information: Domain configure: Disabled Domain auto-reconfigure: Disabled Fabric name: 11:11:11:11:11:11:11:11 Priority: 128 Domain ID: 2 (static) Principal switch running time information: Priority: 128 No interfaces available. The output shows that the domain configuration is complete and that the runtime domain ID of Switch B is 2. Dynamic fabric building configuration example Network requirements As shown in Figure 15, use the dynamic method to build a fabric. 37

44 Figure 15 Network diagram Configuration procedure 1. Configure Switch A: # Configure Switch A to operate in advanced mode, save the configuration, and reboot Switch A. (Skip this step if the switch is operating in advanced mode.) <SwitchA> system-view [SwitchA] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode, and enable the fabric configuration function in VSAN 1 (optional, because this function is enabled by default). <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 1 [SwitchA-vsan1] domain configure enable # Configure the domain ID as 11. [SwitchA-vsan1] domain-id 11 preferred Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchA-vsan1] quit # Create interface VFC 1, and configure it to operate in E mode. [SwitchA] interface vfc 1 [SwitchA-Vfc1] fc mode e # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. [SwitchA-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchA-Vfc1] port trunk vsan 1 [SwitchA-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchA-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 2, and configure it to operate in E mode. 38

45 [SwitchA] interface vfc 2 [SwitchA-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchA-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchA-Vfc2] port trunk vsan 1 [SwitchA-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchA-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 1. [SwitchA] vlan 10 [SwitchA-vlan10] fcoe enable vsan 1 [SwitchA-vlan10] quit 2. Configure Switch B: # Configure Switch B to operate in advanced mode, save the configuration, and reboot Switch B. (Skip this step if the switch is operating in advanced mode.) <SwitchB> system-view [SwitchB] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch B to operate in FCF mode, and enable the fabric configuration function in VSAN 1 (optional, because this function is enabled by default). <SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 1 [SwitchB-vsan1] domain configure enable # Set the switch priority to 1, so that Switch B can be selected as the principal switch. [SwitchB-vsan1] priority 1 [SwitchB-vsan1] quit # Create interface VFC 1, and configure it to operate in E mode. [SwitchB] interface vfc 1 [SwitchB-Vfc1] fc mode e # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. [SwitchB-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchB-Vfc1] port trunk vsan 1 [SwitchB-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/5 [SwitchB-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/5] quit 39

46 # Create interface VFC 2, and configure it to operate in E mode. [SwitchB] interface vfc 2 [SwitchB-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchB-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchB-Vfc2] port trunk vsan 1 [SwitchB-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/6 [SwitchB-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/6] quit # Create interface VFC 3, and configure it to operate in E mode. [SwitchB] interface vfc 3 [SwitchB-Vfc3] fc mode e # Bind interface VFC 3 to interface Ten-GigabitEthernet 1/1/7, and assign it to VSAN 1 as a trunk port. [SwitchB-Vfc3] bind interface ten-gigabitethernet 1/1/7 [SwitchB-Vfc3] port trunk vsan 1 [SwitchB-Vfc3] quit # Assign interface Ten-GigabitEthernet 1/1/7 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/7 [SwitchB-Ten-GigabitEthernet1/1/7] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/7] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/7] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 1. [SwitchB] vlan 10 [SwitchB-vlan10] fcoe enable vsan 1 [SwitchB-vlan10] quit 3. Configure Switch C: # Configure Switch C to operate in advanced mode, save the configuration, and reboot Switch C. (Skip this step if the switch is operating in advanced mode.) <SwitchC> system-view [SwitchC] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch C to operate in FCF mode, and enable the fabric configuration function in VSAN 1 (optional, because this function is enabled by default). <SwitchC> system-view [SwitchB] fcoe-mode fcf [SwitchC] vsan 1 [SwitchC-vsan1] domain configure enable # Configure the domain ID as 13. [SwitchC-vsan1] domain-id 13 preferred 40

47 Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchC-vsan1] quit # Enable LLDP globally. [SwitchC] lldp global enable # Enable LLDP on interface Ten-GigabitEthernet 1/1/7, and enable the interface to advertise DCBX TLVs. [SwitchC] interface ten-gigabitethernet 1/1/7 [SwitchC-Ten-GigabitEthernet1/1/7] lldp enable [SwitchC-Ten-GigabitEthernet1/1/7] lldp tlv-enable dot1-tlv dcbx [SwitchC-Ten-GigabitEthernet1/1/7] quit # Create an Ethernet frame header ACL numbered 4000, and configure two rules to match FCoE frames (protocol type 0x8906) and FIP frames (protocol type 0x8914). [SwitchC] acl number 4000 [SwitchC-acl-ethernetframe-4000] rule permit type 8906 ffff [SwitchC-acl-ethernetframe-4000] rule permit type 8914 ffff [SwitchC-acl-ethernetframe-4000] quit # Create a class named app_c with the logic OR operator, and specify ACL 4000 as the match criterion. [SwitchC] traffic classifier app_c operator or [SwitchC-classifier-app_c] if-match acl 4000 [SwitchC-classifier-app_c] quit # Create a behavior named app_b, and configure the action of marking packets with 802.1p priority 3. [SwitchC] traffic behavior app_b [SwitchC-behavior-app_b] remark dot1p 3 [SwitchC-behavior-app_b] quit # Create a QoS policy named plcy, and associate the class app_c with the behavior app_b in the QoS policy, specifying that the class-behavior association applies only to DCBX. [SwitchC] qos policy plcy [SwitchC-qospolicy-plcy] classifier app_c behavior app_b mode dcbx [SwitchC-qospolicy-plcy] quit # Apply the QoS policy plcy to the outbound direction of interface Ten-GigabitEthernet 1/1/7. [SwitchC] interface ten-gigabitethernet 1/1/7 [SwitchC-Ten-GigabitEthernet1/1/7] qos apply policy plcy outbound [SwitchC-Ten-GigabitEthernet1/1/7] quit # Configure the mapping from 802.1p priority 3 to local precedence 3 in the outbound direction. (This is the default, and you can modify the mapping as needed.) [SwitchC] qos map-table dot1p-lp [SwitchC-maptbl-out-dot1p-lp] import 3 export 3 [SwitchC-maptbl-out-dot1p-lp] quit # Enable byte-count WRR on interface Ten-GigabitEthernet 1/1/7, and configure queue 3 to use SP queuing. [SwitchC] interface ten-gigabitethernet 1/1/7 [SwitchC-Ten-GigabitEthernet1/1/7] qos wrr byte-count [SwitchC-Ten-GigabitEthernet1/1/7] qos wrr 3 group sp 41

48 # Enable interface Ten-GigabitEthernet 1/1/7 to automatically negotiate with its peer to decide whether to enable PFC, enable PFC for 802.1p priority 3, and configure Ten-GigabitEthernet 1/1/7 to trust the 802.1p priority carried in packets. [SwitchC-Ten-GigabitEthernet1/1/7] priority-flow-control auto [SwitchC-Ten-GigabitEthernet1/1/7] priority-flow-control no-drop dot1p 3 [SwitchC-Ten-GigabitEthernet1/1/7] qos trust dot1p [SwitchC-Ten-GigabitEthernet1/1/7] quit # Create interface VFC 1, and configure it to operate in E mode. [SwitchC] interface vfc 1 [SwitchC-Vfc1] fc mode e # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. [SwitchC-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchC-Vfc1] port trunk vsan 1 [SwitchC-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchC] interface ten-gigabitethernet 1/1/5 [SwitchC-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchC-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchC-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 2, and configure it to operate in E mode. [SwitchC] interface vfc 2 [SwitchC-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchC-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchC-Vfc2] port trunk vsan 1 [SwitchC-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchC] interface ten-gigabitethernet 1/1/6 [SwitchC-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchC-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchC-Ten-GigabitEthernet1/1/6] quit # Create interface VFC 3, and configure it to operate in F mode. [SwitchC] interface vfc 3 [SwitchC-Vfc3] fc mode f # Bind interface VFC 3 to interface Ten-GigabitEthernet 1/1/7, and assign it to VSAN 1 as a trunk port. [SwitchC-Vfc3] bind interface ten-gigabitethernet 1/1/7 [SwitchC-Vfc3] port trunk vsan 1 [SwitchC-Vfc3] quit # Assign interface Ten-GigabitEthernet 1/1/7 to VLAN 10 as a trunk port. [SwitchC] interface ten-gigabitethernet 1/1/7 [SwitchC-Ten-GigabitEthernet1/1/7] port link-type trunk [SwitchC-Ten-GigabitEthernet1/1/7] port trunk permit vlan 10 [SwitchC-Ten-GigabitEthernet1/1/7] quit 42

49 # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 1. [SwitchC] vlan 10 [SwitchC-vlan10] fcoe enable vsan 1 [SwitchC-vlan10] quit 4. Configure Switch D: # Configure Switch D to operate in advanced mode, save the configuration, and reboot Switch D. (Skip this step if the switch is operating in advanced mode.) <SwitchD> system-view [SwitchD] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch D to operate in FCF mode, and enable the fabric configuration function in VSAN 1 (optional, because this function is enabled by default). <SwitchD> system-view [SwitchD] fcoe-mode fcf [SwitchD] vsan 1 [SwitchD-vsan1] domain configure enable # Configure the domain ID as 14. [SwitchD-vsan1] domain-id 14 preferred Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchD-vsan1] quit # Enable LLDP globally. [SwitchD] lldp global enable # Enable LLDP on interface Ten-GigabitEthernet 1/1/6, and enable the interface to advertise DCBX TLVs. [SwitchD] interface ten-gigabitethernet 1/1/6 [SwitchD-Ten-GigabitEthernet1/1/6] lldp enable [SwitchD-Ten-GigabitEthernet1/1/6] lldp tlv-enable dot1-tlv dcbx [SwitchD-Ten-GigabitEthernet1/1/6] quit # Create an Ethernet frame header ACL numbered 4000, and configure two rules to match FCoE frames (protocol type 0x8906) and FIP frames (protocol type 0x8914). [SwitchD] acl number 4000 [SwitchD-acl-ethernetframe-4000] rule permit type 8906 ffff [SwitchD-acl-ethernetframe-4000] rule permit type 8914 ffff [SwitchD-acl-ethernetframe-4000] quit # Create a class named app_c with the logic OR operator, and specify ACL 4000 as the match criterion. [SwitchD] traffic classifier app_c operator or [SwitchD-classifier-app_c] if-match acl 4000 [SwitchD-classifier-app_c] quit # Create a behavior named app_b, and configure the action of marking packets with 802.1p priority 3. [SwitchD] traffic behavior app_b [SwitchD-behavior-app_b] remark dot1p 3 [SwitchD-behavior-app_b] quit 43

50 # Create a QoS policy named plcy, and associate the class app_c with the behavior app_b in the QoS policy, specifying that the class-behavior association applies only to DCBX. [SwitchD] qos policy plcy [SwitchD-qospolicy-plcy] classifier app_c behavior app_b mode dcbx [SwitchD-qospolicy-plcy] quit # Apply the QoS policy plcy to the outbound direction of interface Ten-GigabitEthernet 1/1/6. [SwitchD] interface ten-gigabitethernet 1/1/6 [SwitchD-Ten-GigabitEthernet1/1/6] qos apply policy plcy outbound [SwitchD-Ten-GigabitEthernet1/1/6] quit # Configure the mapping from 802.1p priority 3 to local precedence 3 in the outbound direction. (This is the default, and you can modify the mapping as needed.) [SwitchD] qos map-table dot1p-lp [SwitchD-maptbl-out-dot1p-lp] import 3 export 3 [SwitchD-maptbl-out-dot1p-lp] quit # Enable byte-count WRR on interface Ten-GigabitEthernet 1/1/6, and configure queue 3 to use SP queuing. [SwitchD] interface ten-gigabitethernet 1/1/6 [SwitchD-Ten-GigabitEthernet1/1/6] qos wrr byte-count [SwitchD-Ten-GigabitEthernet1/1/6] qos wrr 3 group sp # Enable interface Ten-GigabitEthernet 1/1/6 to automatically negotiate with its peer to decide whether to enable PFC, enable PFC for 802.1p priority 3, and configure Ten-GigabitEthernet 1/1/6 to trust the 802.1p priority carried in packets. [SwitchD-Ten-GigabitEthernet1/1/6] priority-flow-control auto [SwitchD-Ten-GigabitEthernet1/1/6] priority-flow-control no-drop dot1p 3 [SwitchD-Ten-GigabitEthernet1/1/6] qos trust dot1p [SwitchD-Ten-GigabitEthernet1/1/6] quit # Create interface VFC 1, and configure it to operate in E mode. [SwitchD] interface vfc 1 [SwitchD-Vfc1] fc mode e # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. [SwitchD-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchD-Vfc1] port trunk vsan 1 [SwitchD-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchD] interface ten-gigabitethernet 1/1/5 [SwitchD-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchD-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchD-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 2, and configure it to operate in F mode. [SwitchD] interface vfc 2 [SwitchD-Vfc2] fc mode f # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchD-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchD-Vfc2] port trunk vsan 1 [SwitchD-Vfc2] quit 44

51 # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchD] interface ten-gigabitethernet 1/1/6 [SwitchD-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchD-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchD-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 1. [SwitchD] vlan 10 [SwitchD-vlan10] fcoe enable vsan 1 [SwitchD-vlan10] quit Verifying the configurations # Display the domain information of VSAN 1 on Switch A. [SwitchA] display fc domain vsan 1 Domain Information of VSAN 1: Running time information: State: Stable Switch WWN: 48:33:43:2d:46:43:1A:1A Fabric name: 48:33:43:2d:46:43:1B:1B Priority: 128 Domain ID: 11 Configuration information: Domain configure: Enabled Domain auto-reconfigure: Disabled Fabric name: 48:33:43:2d:46:43:1A:1A Priority: 128 Domain ID: 11 (preferred) Principal switch running time information: Priority: 1 Path Upstream Downstream Interface Vfc1 Vfc2 The output shows that the domain configuration is complete and that the principal switch assigns domain ID 11 to Switch A. # Display the domain ID list of VSAN 1 on Switch A. [SwitchA] display fc domain-list vsan 1 Domain list of VSAN 1: Number of domains: 4 Domain ID 0x01(1) 0x0b(11) 0x0d(13) 0x0e(14) WWN 48:33:43:2d:46:43:1B:1B [Principal] 48:33:43:2d:46:43:1A:1A [Local] 48:33:43:2d:46:43:1C:1C 48:33:43:2d:46:43:1D:1D 45

52 The output shows that Switch B becomes the principal switch and assigns the smallest domain ID 1 to itself. 46

53 Configuring VSAN The virtual storage area network (VSAN) technology breaks a physical SAN into multiple VSANs, and provides more secure, reliable, and flexible services. Devices in a VSAN cannot get information about any other VSAN and devices in any other VSAN. Each VSAN performs the following operations independently: selecting a principal switch, assigning domain IDs, running routing protocols, maintaining routing table and FIB table, and providing services. The VSAN technology delivers the following benefits: Improved security VSANs are isolated from each other. Improved adaptability Each VSAN independently runs and provides services. Different VSANs can use the same address space so that network capacity is improved. Flexibility You can assign interfaces to different VSANs without changing the physical connections of the SAN. VSAN fundamentals VFC interfaces on the switch can only work as trunk ports. A trunk port can belong to multiple VSANs. Trunk VSAN in an FC network The trunk VSAN technology implements logical isolation among VSANs. The trunk VSAN works as follows: The trunk VSAN adds a Virtual Fabric Tagging Header (VFT_Header, also known as VSAN tag) to the FC frames. The VFT_Header contains a VF_ID (also known as "VSAN ID") field to indicate the VSAN of the FC frames. In this way, FC frames within different VF_IDs are contained in their respective VSANs, and different VSANs cannot communicate with each other. VSAN tags are added to and removed from an FC frame during transmission. A switch supports multiple VSANs one physical interface, thus reducing physical connections and implementing logical isolation in a physically connected SAN. During the transmission process, VFT_Headers are added to and removed from the frames. A switch can use the same physical interface to support multiple VSANs. The trunk VSAN technology reduces the number of physical connections, actually implementing logical isolation in a physical network. Trunk VSAN in an FCoE network FCoE carries FC over Ethernet. In an FCoE network, VSANs in FC need to be mapped to VLANs as configured by the user, and the FIB table for a VSAN is also stored on the relevant VLAN. FCoE packets use VLAN_Header in place of VFT_Header in FC frames and are forwarded based on the VLAN ID in VLAN_Header. A VFC interface can only work as a trunk port. The bound Ethernet interface must also be configured as a trunk port, and its trunk VLAN list must include the VLANs mapped to each VSAN in the VSAN trunk list of the VFC interface. An FCoE packet transmitted from a VFC interface can use the VLAN ID in VLAN_Header to identify the VLAN to which it belongs. 47

54 Creating a VSAN Initially, only the default VSAN (VSAN 1) exists. You cannot create or delete VSAN 1. You can create VSANs 2 to On a device, you can create up to 16 VSANs, including the default VSAN. To create a VSAN: Step Command Remarks 1. Enter system view. system-view N/A 2. Create a VSAN and enter VSAN view. vsan vsan-id By default, only the default VSAN (VSAN 1) exists. Configuring a trunk VSAN A VFC interface can be assigned to multiple VSANs as a trunk port. If you assign an interface to VSANs as a trunk port multiple times, the final trunk VSAN list is the set of all the VSANs to which you have assigned the interface. To assign a VFC interface to the specified VSANs as a trunk port: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. interface vfc interface-number N/A 3. Assign the VFC interface to the specified VSANs as a trunk port so that the interface allows the specified VSANs to pass through. port trunk vsan vsan-id-list By default, a VFC interface does not belong to any VSAN as a trunk port. You assign a VFC interface to a VSAN that does not exist as a trunk port. Displaying and maintaining VSAN Execute display commands in any view. Task Display the member ports of the VSAN. Command display vsan [ vsan-id ] port-member VSAN configuration example Network requirements As shown in Figure 16, configure the SAN to meet the following requirements: 48

55 Server A can read and write only the data of Disk A and Disk B. Server B can read and write only the data of Disk C. Figure 16 Network diagram Configuration considerations To meet these requirements, divide the SAN into two VSANs, VSAN 10 and VSAN 20. Each VSAN contains a server and disk devices that can exchange data. Configure the two interfaces connecting FCF switch Switch A to the servers to operate in F mode, and assign the two interfaces as trunk ports to VSAN 10 and VSAN 20, respectively. Configure the three interfaces connecting FCF switch Switch B to the disk devices to operate in F mode, and assign the three interfaces as trunk ports to VSAN 10 or VSAN 20. Configure the interfaces connecting Switch A and Switch B to operate in E mode, configure the trunk mode as on for the two interfaces, and assign the interfaces to VSANs 10 and 20 as trunk ports, so that the link between the two FCF switches can send and receive the frames of the two VSANs at the same time. Configuration procedure 1. Configure Switch A: # Configure Switch A to operate in advanced mode, save the configuration, and reboot Switch A. (Skip this step if the switch is operating in advanced mode.) <SwitchA> system-view [SwitchA] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode. Create VSAN s 10 and 20. <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 10 49

56 [SwitchA-vsan10] quit [SwitchA] vsan 20 [SwitchA-vsan20] quit # Enable LLDP globally. [SwitchA] lldp global enable # Enable LLDP on interfaces Ten-GigabitEthernet 1/1/5 and Ten-GigabitEthernet 1/1/6, and enable the interfaces to advertise DCBX TLVs. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] lldp enable [SwitchA-Ten-GigabitEthernet1/1/5] lldp tlv-enable dot1-tlv dcbx [SwitchA-Ten-GigabitEthernet1/1/5] quit [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] lldp enable [SwitchA-Ten-GigabitEthernet1/1/6] lldp tlv-enable dot1-tlv dcbx [SwitchA-Ten-GigabitEthernet1/1/6] quit # Create an Ethernet frame header ACL numbered 4000, and configure two rules to match FCoE frames (protocol type 0x8906) and FIP frames (protocol type 0x8914). [SwitchA] acl number 4000 [SwitchA-acl-ethernetframe-4000] rule permit type 8906 ffff [SwitchA-acl-ethernetframe-4000] rule permit type 8914 ffff [SwitchA-acl-ethernetframe-4000] quit # Create a class named app_c with the logic OR operator, and specify ACL 4000 as the match criterion. [SwitchA] traffic classifier app_c operator or [SwitchA-classifier-app_c] if-match acl 4000 [SwitchA-classifier-app_c] quit # Create a behavior named app_b, and configure the action of marking packets with 802.1p priority 3. [SwitchA] traffic behavior app_b [SwitchA-behavior-app_b] remark dot1p 3 [SwitchA-behavior-app_b] quit # Create a QoS policy named plcy, and associate the class app_c with the behavior app_b in the QoS policy, specifying that the class-behavior association applies only to DCBX. [SwitchA] qos policy plcy [SwitchA-qospolicy-plcy] classifier app_c behavior app_b mode dcbx [SwitchA-qospolicy-plcy] quit # Apply the QoS policy plcy to the outbound direction of Ten-GigabitEthernet 1/1/5 and Ten-GigabitEthernet 1/1/6. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] qos apply policy plcy outbound [SwitchA-Ten-GigabitEthernet1/1/5] quit [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] qos apply policy plcy outbound [SwitchA-Ten-GigabitEthernet1/1/6] quit # Configure the mapping from 802.1p priority 3 to local precedence 3 in the outbound direction. (This is the default, and you can modify the mapping as needed.) [SwitchA] qos map-table dot1p-lp 50

57 [SwitchA-maptbl-out-dot1p-lp] import 3 export 3 [SwitchA-maptbl-out-dot1p-lp] quit # Enable byte-count WRR on Ten-GigabitEthernet 1/1/5 and Ten-GigabitEthernet 1/1/6, and configure queue 3 to use SP queuing. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] qos wrr byte-count [SwitchA-Ten-GigabitEthernet1/1/5] qos wrr 3 group sp [SwitchA-Ten-GigabitEthernet1/1/5] quit [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] qos wrr byte-count [SwitchA-Ten-GigabitEthernet1/1/6] qos wrr 3 group sp [SwitchA-Ten-GigabitEthernet1/1/6] quit # Enable Ten-GigabitEthernet 1/1/5 and Ten-GigabitEthernet 1/1/6 to automatically negotiate with their peers to decide whether to enable PFC, enable PFC for 802.1p priority 3, and configure the two interfaces to trust the 802.1p priority carried in packets. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] priority-flow-control auto [SwitchA-Ten-GigabitEthernet1/1/5] priority-flow-control no-drop dot1p 3 [SwitchA-Ten-GigabitEthernet1/1/5] qos trust dot1p [SwitchA-Ten-GigabitEthernet1/1/5] quit [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] priority-flow-control auto [SwitchA-Ten-GigabitEthernet1/1/6] priority-flow-control no-drop dot1p 3 [SwitchA-Ten-GigabitEthernet1/1/6] qos trust dot1p [SwitchA-Ten-GigabitEthernet1/1/6] quit # Create interface VFC 1, and configure it to operate in F mode. [SwitchA] interface vfc 1 [SwitchA-Vfc1] fc mode f # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 10 as a trunk port. [SwitchA-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchA-Vfc1] port trunk vsan 10 [SwitchA-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchA-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 2, and configure it to operate in F mode. [SwitchA] interface vfc 2 [SwitchA-Vfc2] fc mode f # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 20 as a trunk port. [SwitchA-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchA-Vfc2] port trunk vsan 20 [SwitchA-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 20 as a trunk port. 51

58 [SwitchA] interface ten-gigabitethernet 1/1/6 [SwitchA-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/6] port trunk permit vlan 20 [SwitchA-Ten-GigabitEthernet1/1/6] quit # Create interface VFC 4, and configure it to operate in E mode. [SwitchA] interface vfc 4 [SwitchA-Vfc4] fc mode e # Bind interface VFC 4 to interface Ten-GigabitEthernet 1/1/8, and assign it to VSANs 10 and 20 as a trunk port. [SwitchA-Vfc4] bind interface ten-gigabitethernet 1/1/8 [SwitchA-Vfc4] port trunk vsan [SwitchA-Vfc4] quit # Assign interface Ten-GigabitEthernet 1/1/8 to VLAN 10 and 20 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/8 [SwitchA-Ten-GigabitEthernet1/1/8] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/8] port trunk permit vlan [SwitchA-Ten-GigabitEthernet1/1/8] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 10. [SwitchA] vlan 10 [SwitchA-vlan10] fcoe enable vsan 10 [SwitchA-vlan10] quit # Enable FCoE for VLAN 20 and map VLAN 20 to VSAN 20. [SwitchA] vlan 20 [SwitchA-vlan20] fcoe enable vsan 20 [SwitchA-vlan20] quit 2. Configure Switch B in the same way Switch A is configured. (Details not shown.) Verifying the configurations 1. Verify the configurations on Switch A by displaying member interfaces of all VSANs. [SwitchA] display vsan port-member VSAN 1: Access Ports: Trunk Ports: VSAN 10: Access Ports: Trunk Ports: Vfc1 Vfc4 VSAN 20: Access Ports: Trunk Ports: Vfc2 Vfc4 2. Verify the configurations on Switch B. 52

59 The output on Switch B is the same as that on Switch A. 53

60 Configuring FC routing and forwarding Overview Routing and forwarding in an FC SAN is achieved through FCF switches. When an FCF switch receives a packet, an FCF switch selects an optimal route based on the destination address and forwards the packet to the next FCF switch in the path until the packet reaches the last FCF switch, which forwards the packet to the destination node. Routing provides the path information that guides the forwarding of packets. Routing table and FIB table An FCF switch determines the best routes by using its routing table and sends those routes to the FIB table, which guides packet forwarding. An FCF switch maintains one routing table and one FIB table for each VSAN. Routing table contents The routing table saves the routes discovered by various routing protocols. Routes in a routing table include the following types: Direct routes Routes discovered by link layer protocols Static routes Routes manually configured by the administrator FSPF routes Routes discovered by the Fabric Shortest Path First (FSPF) protocol To display summary information about a routing table, use the display fc routing-table command as follows: <Sysname> display fc routing-table vsan 1 Routing Table: VSAN 1 Destinations : 6 Routes : 6 Destination/mask Protocol Preference Cost Interface 0x020000/8 FSPF Vfc1 0x120000/8 STATIC 10 0 Vfc2 0xfffc01/24 DIRECT 0 0 InLoop0 0xfffffa/24 DIRECT 0 0 InLoop0 0xfffffc/24 DIRECT 0 0 InLoop0 0xfffffd/24 DIRECT 0 0 InLoop0... A route entry includes the following key items: Destination Destination address of an FC frame. mask Together with the destination address, specifies the destination node or the domain address of an FCF switch. A logical AND operation between the destination address and the network mask yields the domain address of the destination node or FCF switch. For example, if the destination address is 0x and the mask is 0xFF0000, the domain address of the destination node or 54

61 FIB table contents FCF switch is 0x A network mask is made up of a certain number of consecutive 1s. It can be expressed in hexadecimal format or by the number of 1s. Protocol Protocol type: DIRECT Direct routes. STATIC Static routes. FSPF FSPF routes. Preference There might be direct routes, static routes, and FSPF routes to the same destination. All of these types of routes are assigned preferences. Direct routes have a preference of 0, static routes have a preference of 10, and FSPF routes have a preference of 20. The optimal route is the one with the highest priority (smallest preference value). Cost Cost of the route. For routes to the same destination and with the same preference, the route with the lowest cost is the optimal one. The cost of direct routes is 0. The costs of static routes and FSPF routes are configurable. Interface Specifies the interface through which a matching FC frame is to be forwarded out of the FCF switch. Each entry in the FIB table specifies which interface a packet destined for a certain destination node or FCF switch should go out through to reach the next hop (the next FCF switch) or the directly-connected destination node. To display FIB table information, use the display fc fib command as follows: <Sysname> display fc fib vsan 1 FC FIB information in VSAN 1: Destination count: 6 FIB entry count: 6 Destination/Mask Interface 0x020000/8 Vfc1 0x120100/8 Vfc2 0xfffc01/24 InLoop0 0xfffffa/24 InLoop0 0xfffffc/24 InLoop0 0xfffffd/24 InLoop0 The key items Destination, Mask, and Interface in an FIB table have the same meanings as those in a routing table. Direct routes The sources of direct routes include well-known addresses and the FC addresses that the local switch assigns to directly-connected N_Ports. The well-known addresses are usually used to access FCF switches. For usage of common well-known addresses, see "Appendix B Well-known fabric addresses." All well-known addresses are added to the routing table as the destination addresses of direct routes. In such a direct route, the destination address is a well-known address, the mask is 0xFFFFFF, and the outgoing interface is InLoop0. 55

62 When an FCF switch assigns FC addresses to the directly connected N_Ports, the FCF switch also adds the direct routes of these addresses to the routing table. In such a direct route, the destination address is an assigned FC address, the mask is 0xFFFFFF, and the outgoing interface is the VFC interface connected to the N_Port. Static routes Static routes are manually configured by the administrator. After you configure a static route, an FC frame to the specified destination is forwarded along the path specified by the administrator. In a simple network, static routes are enough for implementing network connectivity. By correctly setting and using static routes, you can improve network performance and guarantee bandwidth for critical network applications. However, the static routes cannot automatically adapt to network topology changes. When the network fails or the network topology changes, the routes might fail to be reachable, and the network is interrupted. In this case, you must manually modify the static routes. Static routes support equal-cost routes. When you configure multiple equal-cost static routes to the same destination but with different outgoing interfaces, equal-cost routes are generated. FSPF routes Basic concepts As a route selection protocol based on link states, FSPF can automatically calculate the best path between any two switches in a fabric. FSPF has the following characteristics: Can be used for any topology. Supports equal-cost routes. Performs topology calculations on a per-vsan basis. Runs only on E_Ports and provides a loop-free topology. Provides a topology database on each switch to track the state of all links. Uses the Dijkstra algorithm to calculate routes. Provides fast convergence in the event of topology changes. LSDB The link state database (LSDB) is used to store global topology information for switches and link state information of all switches in link state records (LSRs). LSR An LSR describes information about all link states between a switch and its directly connected switches. Each LSR generated by a switch is called an LSR instance. LSRs generated by all switches comprise the LSDB. An LSR contains one or more pieces of link state information, including the following: LSR hold time. Domain ID of the switch advertising the LSR. LSR instance number. Every time an LSR is updated, the instance number increments by 1. 56

63 FSPF packet types How FSPF works Link ID, which identifies a link and includes the domain ID of the switch at the peer end of the link. Source interface and destination interface of the link. Link type, for example, point-to-point connection. Cost for packet transmission over the link. Each link has a different cost. The smaller the cost, the better the link. The route selection algorithm uses this value to determine the best route. The interface cost is configurable. The following protocol packets are used in FSPF: Hello packets Sent periodically to discover and maintain FSPF neighbors. Link state update (LSU) Advertises local link state information in LSRs to the neighboring switches. Link state acknowledgment (LSA) Acknowledges the received LSR. After receiving an LSU, a switch needs to acknowledge its LSR with an LSA. Otherwise, the neighboring switch retransmits the LSR. FSPF works as follows: 1. The switch periodically sends hello packets to establish neighbor relationships with other switches. 2. After establishing neighbor relationships, the switches synchronize LSDBs by exchanging all LSRs in their respective LSDBs. A switch carries LSRs in LSUs and acknowledges received LSRs with LSAs. 3. After the synchronization is complete, the LSDB in each switch contains LSRs of all switches in the fabric. 4. The switch uses the Dijkstra algorithm to calculate the shortest paths to other switches based on the local LSDB. Then, it determines the outgoing interfaces and generates an FSPF routing table. 5. When the network topology or link state changes, the switch floods a new LSR to its neighboring switches. After receiving the LSR, the neighboring switches add it to their LSDBs and flood it to their respective neighbors. In this way, all switches in the fabric receive that LSR. 6. Local LSDB updating results in SPF calculation. The calculated shortest path tree list is updated to the FSPF routing table. Configuring static routes for FC Configuration restrictions and guidelines The destination address of a static FC route is in the range of to EFFFFF (hexadecimal). You cannot configure a route with a well-known address as the destination address. The outgoing interface of a static FC route can only be a VFC interface. If you configure two routes with the same destination address, mask, and outgoing interface, but with different costs, the route configured later applies. The maximum number of static routes allowed in a VSAN is

64 Configuration procedure To configure a static FC route: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure a static FC route. fc route-static fcid { mask mask-length } interface-type interface-number [ cost cost-value ] By default, no static FC route exists. Configuring FSPF FSPF is enabled by default. Generally, you do not need to make special configurations. You can change FSPF parameters on a per-vsan or per-interface basis as needed. FSPF configuration task list Tasks at a glance Change FSPF parameters for a VSAN in VSAN view: (Required.) Enabling FSPF (Optional.) Configuring the shortest SPF calculation interval (Optional.) Configuring the minimum LSR receiving interval (Optional.) Configuring the minimum LSR refresh interval (Optional.) Change FSPF parameters for an interface in E_Port interface view: Configuring the FSPF cost for an interface Configuring the hello interval for an interface Configuring the dead interval for an interface Configuring the LSR retransmission interval for interfaces Disabling FSPF for an interface (Optional.) Configuring FSPF GR in system view: Configuring the GR restarter Configuring the GR helper Enabling FSPF FSPF-related functions can work in a VSAN only after you enable FSPF for the VSAN. To enable FSPF: Step Command Remarks 1. Enter system view. system-view N/A 58

65 Step Command Remarks 2. Enter VSAN view. vsan vsan-id N/A 3. Enable FSPF for the VSAN. fspf enable By default, FSPF is enabled after a VSAN is created. Configuring the shortest SPF calculation interval When the LSDB changes, SPF calculations occur, which consume CPU resources. To prevent frequent SPF calculations from consuming too many CPU resources, you can configure the shortest SPF calculation interval. The shortest SPF calculation interval defines the minimum interval between two consecutive SPF calculations. A smaller value means that FSPF responds faster to fabric changes by recalculating routes in a VSAN, but it requires more CPU resources. To configure the shortest SPF calculation interval: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure the shortest SPF calculation interval. fspf spf-hold-time value The default setting is 0 second. Configuring the minimum LSR receiving interval The minimum LSR receiving interval specifies the time between receiving LSRs in a VSAN. Any LSR instances of the same LSR received within this time are dropped. This helps avoid frequent SPF calculations caused by LSDB updating. To configure the minimum LSR receiving interval: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure the minimum LSR receiving interval. fspf min-ls-arrival value The default setting is 1 second. Configuring the minimum LSR refresh interval The minimum LSR refresh interval specifies the interval at which LSRs are refreshed. To reduce SPF calculations and LSR flooding in a fabric caused by frequent LSR refreshing, the switch cannot refresh local LSRs within this interval. To configure the minimum LSR refresh interval: 59

66 Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VSAN view. vsan vsan-id N/A 3. Configure the minimum LSR refresh interval. fspf min-ls-interval value The default setting is 5 seconds. Configuring the FSPF cost for an interface Each link has a different cost. The route selection algorithm uses this value to determine the best route. The smaller the interface FSPF cost, the smaller the link cost. To configure the interface FSPF cost: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. 3. Configure the FSPF cost for the VFC interface in a specified VSAN. interface vfc interface-number fspf cost value vsan vsan-id N/A The default setting is 100. Configuring the hello interval for an interface The hello interval specifies the time between the hello packets sent periodically by the switch to discover and maintain neighbor relationships. NOTE: The configured hello interval must be smaller than the dead interval and must be the same at the two ends of the link. To configure the interface hello interval: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. interface vfc interface-number N/A 3. Configure the hello interval for the VFC interface in a specified VSAN. fspf hello-interval value vsan vsan-id The default setting is 20 seconds. Configuring the dead interval for an interface After two switches establish a neighbor relationship, they send hello packets at the hello interval to each other to maintain the neighbor relationship. The dead interval specifies the interval during which at least one hello packet must be received from a neighbor before the neighbor is considered to be nonexistent and is removed. 60

67 NOTE: The configured dead interval must be greater than the hello interval and must be the same at the two ends of the link. To configure the interface dead interval: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. interface vfc interface-number N/A 3. Configure the dead interval for the VFC interface in a specified VSAN. fspf dead-interval value vsan vsan-id The default setting is 80 seconds. Configuring the LSR retransmission interval for interfaces The LSR retransmission interval specifies the time to wait for an LSR acknowledgement from the neighbor before retransmitting the LSR. To configure the LSR retransmission interval: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. interface vfc interface-number N/A 3. Configure the LSR retransmission interval for the VFC interface in a specified VSAN. fspf retransmit-interval value vsan vsan-id The default setting is 5 seconds. Disabling FSPF for an interface With FSPF enabled, an interface can participate in SPF calculation. To avoid SPF calculations on an interface, disable FSPF on the interface. To disable FSPF on an interface: Step Command Remarks 1. Enter system view. system-view N/A 2. Enter VFC interface view. interface vfc interface-number N/A 3. Enable FSPF for the VFC interface in a specified VSAN. fspf silent vsan vsan-id By default, FSPF is enabled on all VFC interfaces. 61

68 Configuring FSPF GR FSPF Graceful Restart (GR) enables nonstop forwarding of traffic by backing up FSPF configuration information during a protocol restart (for example, the FSPF process restart triggered by the process command) or active/standby switchover. GR involves the following roles: GR restarter GR-capable device where a protocol restart or active/standby switchover occurs GR helper The GR restarter's neighboring device that assists in the GR process Configuring the GR restarter Step Command Remarks 1. Enter system view. system-view N/A 2. Enable FSPF GR. fspf graceful-restart By default, FSPF GR is disabled. 3. Configure the maximum interval for FSPF GR. fspf graceful-restart interval interval-value The default setting is 120 seconds. Configuring the GR helper Step Command Remarks 1. Enter system view. system-view N/A 2. Enable FSPF GR helper. fspf graceful-restart helper By default, FSPF GR helper is enabled. Displaying and maintaining FC routing and forwarding Execute display commands in any view. Task Display FC routing table information. Display FC FIB table information. Display FC Exchange table information Command display fc routing-table [ vsan vsan-id ] [ statistics verbose ] display fc routing-table vsan vsan-id fc-id [ mask mask-length ] [ verbose ] display fc fib [ fcid [ mask-length ] ] vsan vsan-id display fc exchange { link protocol } [ slot slot-number ] display fc exchange link verbose [ slot slot-number [ exid exid ] ] Display FSPF neighbor information. display fspf neighbor [ vsan vsan-id ] Display link state database information. display fspf lsdb [ vsan vsan-id ] Display FSPF GR state information. display fspf graceful-restart [ vsan vsan-id ] Display FSPF statistics. display fspf statistics [ vsan vsan-id ] 62

69 Task Command Clear FSPF statistics. reset fspf counters [ vsan vsan-id ] Static FC routing configuration example Network requirements As shown in Figure 17, configure static routes to enable any two FCF switches to communicate with each other. Figure 17 Network diagram Switch B Domain ID: 2 VFC1 XGE1/1/5 VFC2 XGE1/1/6 VFC1 XGE1/1/5 VFC2 XGE1/1/6 Switch A Switch C Domain ID: 1 Domain ID: 3 Configuration procedure 1. Configure Switch A: # Configure Switch A to operate in advanced mode, save the configuration, and reboot Switch A. (Skip this step if the switch is operating in advanced mode.) <SwitchA> system-view [SwitchA] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode. Enable the fabric configuration function in VSAN 1. <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 1 [SwitchA-vsan1] domain configure enable # Configure the domain ID as 1. [SwitchA-vsan1] domain-id 1 static Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchA-vsan1] quit 63

70 # Create interface VFC 1, and configure it to operate in E mode. [SwitchA] interface vfc 1 [SwitchA-Vfc1] fc mode e # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. [SwitchA-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchA-Vfc1] port trunk vsan 1 [SwitchA-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchA-Ten-GigabitEthernet1/1/5] quit # Enable FCoE for VLAN 10 and bind VLAN 10 to VSAN 1. [SwitchA] vlan 10 [SwitchA-vlan10] fcoe enable vsan 1 [SwitchA-vlan10] quit # Configure two static routes. [SwitchA] vsan 1 [SwitchA-vsan1] fc route-static vfc 1 [SwitchA-vsan1] fc route-static vfc 1 [SwitchA-vsan1] quit 2. Configure Switch B: # Configure Switch B to operate in advanced mode, save the configuration, and reboot Switch B. (Skip this step if the switch is operating in advanced mode.) <SwitchB> system-view [SwitchB] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode. Enable the fabric configuration function in VSAN 1. <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 1 [SwitchA-vsan1] domain configure enable # Configure the domain ID as 2. [SwitchB-vsan1] domain-id 2 static Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchB-vsan1] quit # Create interface VFC 1, and configure it to operate in E mode. [SwitchB] interface vfc 1 [SwitchB-Vfc1] fc mode e # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 1 as a trunk port. 64

71 [SwitchB-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchB-Vfc1] port trunk vsan 1 [SwitchB-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/5 [SwitchB-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/5] quit # Create interface VFC 2, and configure it to operate in E mode. [SwitchB] interface vfc 2 [SwitchB-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchB-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchB-Vfc2] port trunk vsan 1 [SwitchB-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/6 [SwitchB-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 10 and bind VLAN 10 to VSAN 1. [SwitchB] vlan 10 [SwitchB-vlan10] fcoe enable vsan 1 [SwitchB-vlan10] quit # Configure two static routes. [SwitchB] vsan 1 [SwitchB-vsan1] fc route-static vfc 1 [SwitchB-vsan1] fc route-static vfc 2 [SwitchB-vsan1] quit 3. Configure Switch C: # Configure Switch C to operate in advanced mode, save the configuration, and reboot Switch C. (Skip this step if the switch is operating in advanced mode.) <SwitchC> system-view [SwitchC] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode. Enable the fabric configuration function in VSAN 1. <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 1 [SwitchA-vsan1] domain configure enable # Configure the domain ID as 3. [SwitchC-vsan1] domain-id 3 static 65

72 Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y [SwitchC-vsan1] quit # Create interface VFC 2, and configure it to operate in E mode. [SwitchC] interface vfc 2 [SwitchC-Vfc2] fc mode e # Bind interface VFC 2 to interface Ten-GigabitEthernet 1/1/6, and assign it to VSAN 1 as a trunk port. [SwitchC-Vfc2] bind interface ten-gigabitethernet 1/1/6 [SwitchC-Vfc2] port trunk vsan 1 [SwitchC-Vfc2] quit # Assign interface Ten-GigabitEthernet 1/1/6 to VLAN 10 as a trunk port. [SwitchC] interface ten-gigabitethernet 1/1/6 [SwitchC-Ten-GigabitEthernet1/1/6] port link-type trunk [SwitchC-Ten-GigabitEthernet1/1/6] port trunk permit vlan 10 [SwitchC-Ten-GigabitEthernet1/1/6] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 1. [SwitchC] vlan 10 [SwitchC-vlan10] fcoe enable vsan 1 [SwitchC-vlan10] quit # Configure two static routes. [SwitchC] vsan 1 [SwitchC-vsan1] fc route-static vfc 2 [SwitchC-vsan1] fc route-static vfc 2 [SwitchC-vsan1] quit Verifying the configurations # Display the FC routing table in VSAN 1 on Switch A. [SwitchA] display fc routing-table vsan 1 Routing Table: VSAN 1 Destinations : 6 Routes : 6 Destination/mask Protocol Preference Cost Interface 0x020000/8 STATIC 10 0 Vfc1 0x030000/8 STATIC 10 0 Vfc1 0xfffc01/24 DIRECT 0 0 InLoop0 0xfffffa/24 DIRECT 0 0 InLoop0 0xfffffc/24 DIRECT 0 0 InLoop0 0xfffffd/24 DIRECT 0 0 InLoop0 # Display the FC routing table in VSAN 1 on Switch B. [SwitchB] display fc routing-table vsan 1 Routing Table: VSAN 1 Destinations : 6 Routes : 6 Destination/mask Protocol Preference Cost Interface 0x010000/8 STATIC 10 0 Vfc1 0x030000/8 STATIC 10 0 Vfc2 0xfffc02/24 DIRECT 0 0 InLoop0 66

73 0xfffffa/24 DIRECT 0 0 InLoop0 0xfffffc/24 DIRECT 0 0 InLoop0 0xfffffd/24 DIRECT 0 0 InLoop0 # Display the FC routing table in VSAN 1 on Switch C. [SwitchC] display fc routing-table vsan 1 Routing Table: VSAN 1 Destinations : 6 Routes : 6 Destination/mask Protocol Preference Cost Interface 0x010000/8 STATIC 10 0 Vfc2 0x020000/8 STATIC 10 0 Vfc2 0xfffc03/24 DIRECT 0 0 InLoop0 0xfffffa/24 DIRECT 0 0 InLoop0 0xfffffc/24 DIRECT 0 0 InLoop0 0xfffffd/24 DIRECT 0 0 InLoop0 # On Switch A, use the fcping command to ping Switch C and check whether Switch C is reachable. [SwitchA] fcping fcid fffc03 vsan 1 FCPING fcid 0xfffc03: 128 data bytes, press CTRL_C to break Reply from 0xfffc03: bytes = 128 time = 23 ms Reply from 0xfffc03: bytes = 128 time = 9 ms Reply from 0xfffc03: bytes = 128 time = 19 ms Reply from 0xfffc03: bytes = 128 time = 14 ms Reply from 0xfffc03: bytes = 128 time = 25 ms --- 0xfffc03 fcping statistics packet(s) transmitted 5 packet(s) received 0.00% packet loss round-trip min/avg/max = 9/18/25 ms The output shows that Switch A can reach Switch C. FSPF configuration example Network requirements As shown in Figure 18, configure FSPF to enable the two FCF switches to communicate with each other. Figure 18 Network diagram Configuration procedure 1. Configure Switch A: 67

74 # Configure Switch A to operate in advanced mode, save the configuration, and reboot Switch A. (Skip this step if the switch is operating in advanced mode.) <SwitchA> system-view [SwitchA] system-working-mode advance Do you want to change the system working mode? [Y/N]:y The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch A to operate in FCF mode. Enable the fabric configuration function in VSAN 2. <SwitchA> system-view [SwitchA] fcoe-mode fcf [SwitchA] vsan 2 [SwitchA-vsan2] domain configure enable # Configure the domain ID as 1. [SwitchA-vsan2] domain-id 1 static Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y # Enable FSPF globally. [SwitchA-vsan2] fspf enable [SwitchA-vsan2] quit # Create interface VFC 1, and configure it to operate in E mode. [SwitchA] interface vfc 1 [SwitchA-Vfc1] fc mode e # Bind VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 2 as a trunk port. [SwitchA-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchA-Vfc1] port trunk vsan 2 [SwitchA-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchA] interface ten-gigabitethernet 1/1/5 [SwitchA-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchA-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchA-Ten-GigabitEthernet1/1/5] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 2. [SwitchA] vlan 10 [SwitchA-vlan10] fcoe enable vsan 2 [SwitchA-vlan10] quit # Enable FSPF for interface VFC 1. [SwitchA] interface vfc 1 [SwitchA-Vfc1] undo fspf silent vsan 2 [SwitchA-Vfc1] quit 2. Configure Switch B: # Configure Switch B to operate in advanced mode, save the configuration, and reboot Switch B. (Skip this step if the switch is operating in advanced mode.) <SwitchB> system-view [SwitchB] system-working-mode advance Do you want to change the system working mode? [Y/N]:y 68

75 The system working mode is changed, please save the configuration and reboot the system to make it effective. # Configure Switch B to operate in FCF mode. Enable the fabric configuration function in VSAN 2. <SwitchB> system-view [SwitchB] fcoe-mode fcf [SwitchB] vsan 2 [SwitchB-vsan2] domain configure enable # Configure the domain ID as 2. [SwitchB-vsan2] domain-id 2 static Non-disruptive reconfiguration or isolating the switch may be performed. Continu e? [Y/N]:y # Enable FSPF globally. [SwitchB-vsan2] fspf enable [SwitchB-vsan2] quit # Create interface VFC 1, and configure it to operate in E mode. [SwitchB] interface vfc 1 [SwitchB-Vfc1] fc mode e # Bind interface VFC 1 to interface Ten-GigabitEthernet 1/1/5, and assign it to VSAN 2 as a trunk port. [SwitchB-Vfc1] bind interface ten-gigabitethernet 1/1/5 [SwitchB-Vfc1] port trunk vsan 2 [SwitchB-Vfc1] quit # Assign interface Ten-GigabitEthernet 1/1/5 to VLAN 10 as a trunk port. [SwitchB] interface ten-gigabitethernet 1/1/5 [SwitchB-Ten-GigabitEthernet1/1/5] port link-type trunk [SwitchB-Ten-GigabitEthernet1/1/5] port trunk permit vlan 10 [SwitchB-Ten-GigabitEthernet1/1/5] quit # Enable FCoE for VLAN 10 and map VLAN 10 to VSAN 2. [SwitchA] vlan 10 [SwitchA-vlan10] fcoe enable vsan 2 [SwitchA-vlan10] quit # Enable FSPF for interface VFC 1. [SwitchB] interface vfc 1 [SwitchB-Vfc1] undo fspf silent vsan 2 [SwitchB-Vfc1] quit Verifying the configurations # Display FSPF neighbor information on Switch A. [SwitchA] display fspf neighbor FSPF neighbor information of VSAN 2(01): Interface NbrDomain IfIndex NbrIfIndex Dead Time State Vfc1 2 0x68 0x68 00:01:06 Full # Display information about the FC routing table on Switch A. [SwitchA] display fc routing-table vsan 2 69

76 Routing Table: VSAN 2 Destinations : 5 Routes : 5 Destination/mask Protocol Preference Cost Interface 0x020000/8 FSPF Vfc1 0xfffc01/24 DIRECT 0 0 InLoop0 0xfffffa/24 DIRECT 0 0 InLoop0 0xfffffc/24 DIRECT 0 0 InLoop0 0xfffffd/24 DIRECT 0 0 InLoop0 # On Switch A, use the fcping command to ping Switch B and check whether Switch B is reachable. [SwitchA] fcping fcid fffc02 vsan 2 FCPING fcid 0xfffc02: 128 data bytes, press CTRL_C to break. Reply from 0xfffc02: bytes = 128 time = ms Reply from 0xfffc02: bytes = 128 time = ms Reply from 0xfffc02: bytes = 128 time = ms Reply from 0xfffc02: bytes = 128 time = ms Reply from 0xfffc02: bytes = 128 time = ms --- 0xfffc02 fcping statistics packet(s) transmitted 5 packet(s) received 0.00% packet loss round-trip min/avg/max = 0.247/0.430/1.102 ms The output shows that Switch A can reach Switch B. 70

77 Configuring FC zones Overview The VSAN technology divides a physical SAN into multiple VSANs, which are separated from one another, and provides more secure, reliable, and flexible services. A VSAN, however, cannot perform access control over the servers and disk devices (or the N_Ports) connected to a fabric. N_Ports in the same VSAN can access one another only if these N_Ports register name services. This creates data security risks. Zoning can solve the preceding problem by dividing a VSAN into zones and adding N_Ports to different zones for different purposes. In this manner, N_Ports in different zones are separated to implement access control. Zone database To control access among N_Ports, you can divide N_Ports into different zones as needed, which comprise a zone set. The same N-Ports can form multiple zone sets according to different zone division policies. These zones and zone sets comprise a zone database. Zone database structure The zone database is organized into three levels: zone set, zone, and zone member. Figure 19 Zone database structure In the zone database structure: A zone set is a set of zones. A zone is a set of zone members, which are N_Ports. Zone membership can be identified by the port WWN (pwwn) or FC address of an N_Port. 71

78 Active zone set Each VSAN can have multiple zone sets, each zone set can have multiple zones, and each zone can have multiple zone members. To facilitate configuration, zone membership configuration supports use of zone aliases. A zone alias is a set of N_Ports, which can be considered as a whole. You can add common zone members in multiple zones to a zone alias, and call the zone alias in different zones to simplify configuration. Each VSAN can have multiple zone sets, but only one zone set can be effective at a time. It is called the "active zone set." Access control over N_Ports is subject to the active zone set. To ensure consistent access control over N_Ports on a fabric-wide basis, you must specify the active zone set by using a command on a local device and distribute it to the entire fabric. When you activate a zone set, a copy of the zone set at the time of activation is created and is called the active zone set. After that, modifications to the zone set do not take effect on the copy until the copy is reactivated. Figure 20 shows the relationship between active and full zone sets. Figure 20 Active and full zone sets 72

79 Default zone The N_Ports in zones of the active zone set are part of the active zone set. Registered N_Ports that are not in the active zone set automatically become part of the default zone. If members of the default zone are allowed to access each other, the default zone can be considered to be part of the active zone set, and it participates in access control among N_Ports. Otherwise, the default zone is not in the active zone set and does not participate in access control among the N_Ports. Distributing zones Distributing zones indicates that a device distributes its active zone set or zone database to all the other devices in the same fabric. The distributing device is called a "manager switch," and all the other devices are called "managed switches." The following distribution types are provided: Complete distribution Distributes both the active zone set and zone database. Incomplete distribution Distributes only the active zone set. Methods of triggering a distribution Trigger a distribution by using one of the following methods: Activate a zone set as the active zone set on a switch by using the zoneset activate command. At the time of activation, the active zone set is distributed to all the other switches. This method determines whether to carry the zone database according to the configured distribution type. Distribute the active zone set and zone database directly by using the zoneset distribute command on a switch. This method performs a complete distribution irrespective of the configured distribution type. Managed switches replace their respective active zone sets or zone databases with the received data, regardless of the distribution types configured on them. For example, if a managed switch receives the active zone set and zone database, it replaces its local active zone set and zone database with them regardless of whether its distribution type is complete distribution. Zone distribution process The manager switch completes data synchronization with each managed switch by using the following packets: Acquire Change Authorization (ACA) Stage Fabric Configuration Update (SFC) Update Fabric Configuration (UFC) Release Change Authorization (RCA) These types of packets implement locking, data synchronization, submission, and unlocking processes, respectively. These processes make sure only one device distributes data as the manager switch when multiple users trigger a data distribution by using commands on different devices at the same time. 73

80 Figure 21 Distribution process The distribution process is as follows: 1. The manager switch obtains the status of each managed switch through an ACA request, which carries the fabric-wide list of domain IDs (addresses of all switches in the fabric) known to the manager switch. After sending the ACA request, the manager switch enters the locked state. After receiving the ACA request, a managed switch compares its list of domain IDs with that in the packet. If they are consistent, the fabric is in stable state [1]. In this case, the managed switch is prepared for synchronization, replies with an ACC (acknowledgement) packet, and enters the change authorization state (locked). If the managed switch has been in change authorization state or cannot process the ACA request for some reason, it replies with an RJT (reject) packet. 2. The manager switch starts data synchronization by sending an SFC request only after receiving ACC requests from all managed switches. Otherwise, it notifies managed switches to release the change authorization state by sending an RCA request [2]. 3. The manager switch sends an SFC request to all managed switches. The SFC request carries data to be synchronized, including the active zone set and zone database information. After receiving the SFC request with zone database information, the managed switch determines whether the total number of zones, zone sets, and zone aliases exceeds the limit after its local zone database is replaced. If not, it replies with an ACC packet. Otherwise, it replies with an RJT packet. 4. The manager switch notifies managed switches by sending a UFC request to replace their local data with the received data only after receiving ACC packets from all managed switches. Otherwise, it notifies managed switches to release the change authorization state by sending an RCA request. 5. After receiving the UFC request, the managed switch updates its local zone database. It replies with an ACC packet for a successful update and with an RJT packet for a failed update. 6. The manager switch notifies managed switches by sending an RCA request to release the change authorization state after receiving ACC packets from all managed switches. 74

81 7. After receiving the RCA request, the managed switch releases its change authorization state and replies with an ACC packet. 8. The manager switch releases its change authorization state after receiving ACC packets from all managed switches. NOTE: [1] This actually requires the routing information across the fabric to be correct and consistent and eliminating unreachable routes. You need to pay special attention to this in the case of using static routes. Otherwise, data cannot be correctly distributed. [2] If the managed switch replies with neither an ACC packet nor an RJT packet because of its abnormal state, the manager switch cannot be released from its locked state. To prevent this situation, the manager switch starts a packet retransmission mechanism, transmitting up to three ACA requests. In this case, if no reply is received, the manager switch releases its locked state. If the manager switch becomes abnormal after sending an ACA request, the managed switch will be in locked state but cannot receive subsequent packets. Similarly, the managed switch releases its locked state after waiting for a period of time. Zone merge When two fabrics are merged, both the active zone set and zone database might exist in each fabric. In this case, zone configuration information needs to be merged. The following merge types are provided: Complete merge Merges both the active zone sets and zone databases. Incomplete merge Merges only the active zone sets. The merge-originating switch checks its local merge type. If it is configured with complete merge, it sends packets with both the active zone set and zone database. Otherwise, it sends packets with only the active zone set. NOTE: The merge-originating switch determines the data to be merged according to its locally configured merge type, while the merged switches merge all received data, regardless of the their merge types. The pwwn is a preferred choice over FC addresses to identify zone members, because FC addresses might change at fabric merge and the merge result might not be as expected by users. Zone merge process When a switch discovers a new neighbor (the link layer module discovers neighbors and notifies the zone module), it starts a merge process with the neighbor. If the data changes after merging, the switch sends the changed data to neighbor switches until all switches in the fabric update their data. During the merge, the switch sends Merge Request Resource Allocation (MRRA) requests to negotiate the size of data transmitted and then Merge Request (MR) packets containing data to be merged to neighbor switches. 75

82 Figure 22 Zone merge process between two switches The zone merge process is as follows: 1. Switch A and Switch B are new neighbors to each other. Suppose that Switch A first initiates a merge to Switch B: a. Switch A sends an MRRA request carrying the size of its data to be merged to Switch B. b. After receiving the MRRA request, Switch B determines whether to accept the merge according to its local data size. If the size of the data to be merged is acceptable, it replies with an ACC packet. Otherwise, it replies with an RJT packet. c. After receiving the ACC packet, Switch A sends an MR request containing its zone data to Switch B. d. After receiving the MR request, Switch B obtains the zone data and merges it with its local zone data. Then, it replies with an ACC packet for a successful merge or with an RJT packet containing the cause of failure for a failed merge. 2. After the merge process initiated by Switch A is complete, Switch B ends the merge process with Switch A if its local data is exactly the same as or a subset of that of Switch A. Otherwise, Switch B initiates a merge process with Switch A, which is similar to that initiated by Switch A to Switch B as shown in steps 5, 6, 7, and 8 in Figure After the merge process initiated by Switch A is complete, Switch B synchronizes changes in its local database arising from the merge to the entire fabric by initiating a merge process to all its neighbors. 4. Two 1-way merge processes can ensure data consistency between Switch A and Switch B. 76