Revision 1009 1
Revision 1009 2
Footnote 1: FC_ID is also known as a 24-bit Fibre Channel address. Revision 1009 3
In a native Fibre Channel environment, a node port is directly connected to a fabric port on a switch, and some switch ports may be connected to other switches. To login, the node port simply sends an FLOGI request to the fabric port on the switch. In FCoE, a node port may not be directly connected to an FCoE fabric port, but to one or more CEE switches. As a result, an FCoE node ports needs a discovery mechanism that allows it to discover any available fabric ports before it can perform an FLOGI. Similarly, FCoE switch E_Ports may not be directly connected to other E_Ports in the same manner as they are in a Fibre Channel environment. This requires a discovery mechanism that allows one switch port to discover other switch ports before an ISL can be initialized. The mechanism used for both types of discovery is the FCoE Initialization Protocol (FIP). FIP uses EtherType 0x8914. Revision 1009 4
Revision 1009 5
Footnote 1: Fabric OS v6.1.2_cee supports FC-BB-5 version 1.01 (June 2008) of the FC-BB-5 standards. This version is referred to as Pre-FIP by the standards committee. Fabric OS v6.1.2_cee1 and v6.3.0 support both the Pre-FIP and Fip (version 0) version 1.03 Footnote 2: This is how the Brocade-branded CNAs (BR-1010 and BR-1020) act; other CNAs may do something different, it would depend on how the CNA driver was written. Example : Qlogic Gen2 does the following: Relies on the FCoE app TLV to start the discovery/login procedure. Withdrawal of this TLV will cause a logout. Starts with FIP solicitation. If this fails, keeps soliciting till it passes. Once the solicitation passes, it tries FIP FLOGI. If the FIP FLOGI fails, it tries 8 times, then goes back to soliciting. It never falls back to pre-fip. Revision 1009 6
Revision 1009 7
Footnote 1: FCOE Encapsulation frames are used in the data transfer phase. Footnote 2: FIP frames carry Discovery, Link Initialization and port instantiation services. Revision 1009 8
The FCoE Initialization Protocol frame format is a traditional Ethernet frame with payload contents that are specific for FIP. The Destination and Source MAC address are the first two fields. Just after Word 2 (SA), you may find an optional IEEE 802.1Q VLAN tag. This tag would be required in a CEE environment. Following the tag word is the Ethertype field which would contain a value 0x8914 indicating that the frame is a FIP frame. That is followed by the Version field, which specifies which FIP Frame version this frame is. In the next word (word 4) we see the FIP Operation Code and a FIP Subcode. The FIP OpCode Describes the frame Operation (such as Discovery or Virtual link Initialization), and the Subcode provides additional detail regarding the OpCode. The Descriptor length indicates how many descriptors are in the Descriptor list. That is followed by 5 flags. Two of the Flags that are currently defined include FP and SP, which represents the FPMA and SPMA MAC Address Assignment respectively. The Descriptor List provides parameters associated with the FIP Operations. After that comes a FIP PAD field for use with Mini jumbo frames, or if the frame is not long enough. The last word contains the Ethernet Frame Check Sequence which is a checksum value representing the contents of the frame. Revision 1009 9
Above is a list of the FIP Operation Codes. Each of the Operation Codes have associated Subcodes which further define the actual FIP operation being performed. These Operation Codes and Subcodes values would be carried in word 4 of the FIP frame as diagramed on the previous slide. An example is OpCode value 0001 hex with subcodes 01h an 02h represent the operations Discovery, Solicitation and Discovery, Advertisement respectively. It is expected that other Opcode / subcodes will be defined before the standard is actually completed. Toward the bottom of the table you will see the Vendor Specific OP Codes which range from FFF8h-FFFEh. These Vendor Specific OpCoded are not recommended for usage because this typically results in a propriety implementation that is not interoperable with other vendors. All other OpCode values are Reserved. Revision 1009 10
ENode - is an FCoE End Node with one or more Converged Network Adapters (CNAs) FCF Fibre Channel Forwarder FLOGI Fabric LOGIn VN_Port Virtual N_Port FPMA Fabric Provided MAC Address SPMA Server Provided MAC Address Revision 1009 11
In this multi-slide example, Host 2 (H2) generates a multicast solicitation frame destined to all FCF MAC addresses to identify all FCFs in the network. This frame carries its own MAC address in the Source MAC field. In the lower portion of the diagram is the FIP frame with some of the fields defined: DA = All FCF MACs; SA of H2; OpCode and Subcode indicates Discovery, Solicitation, and then the Descriptor list of the capabilities of the ENode (H2), including this frame initiated at a ENode; addressing mode (SPMA/FPMA it supports; the H2 MAC ADDR; and the Port ID, and last is the Maximum FCOE Frame size supported, and generally there will be at least support for mini-jumbo frames. This frame is received by all the FCF ports in the network (FCF A and B). Revision 1009 12
Next, a unicast Advertisement is sent from each of the FCFs using their MAC addresses as the Source address of the frame, and the Host 2 MAC address as the Destination address. These two Advertisement frames can be viewed in the lower center of the diagram. The first entry in brackets in the Descriptor list is Solicited, which indicates that this frame is a response to a solicitation from Host 2. Additionally, we see that the frame is coming From (an) FCF, and which address modes are supported. The Priority provides the ability to designate one of several FCFs that may exist in the fabric as the preferred FCF. Next we see the MAC of the FCF who is responding, followed by the Fibre Channel Switch_Name and the Fabric_Name. This FC Fabric includes the two FC forwarder switches. Revision 1009 13
At this time Host 2 is ready to login to the FC Fabric. Host 2 will generate a FIP frame, as shown in the bottom center of the diagram, destined to FCF_A for our example. This frame carries a standard FC Login request (FLOGI) as one of the Descriptors. Other descriptor field parameters include ENODE that the frame is from, what address modes are supported, and proposed MAC address, especially if it is an SPMA assigned address. Revision 1009 14
The FCF will reply to the host with a FIP frame that contains a standard FLOGI response as one of the descriptors. The other descriptors carry parameters indicating that the response is from an FCF, that an SPMA or FPMA address has been assigned, and it assigns a MAC address that is tied to the FC_ID that is assigned to the End port (H2) in the FLOGI response information. Host 2 now has an assigned MAC address and a 24 bit FC address. Revision 1009 15
The resulting virtual FC link between H2 and FCF_A runs standard FCoE traffic across the link. Revision 1009 16
Revision 1009 17
Support for the June 10, 2008 version for Fabric OS v6.1.1_cee release, and the October 9, 2008 version for Fabric OS v6.1.2_cee1. We will silently drop FIP discovery frames complying to the Oct 9 th version if running Fabric OS v6.1.2_cee. Revision 1009 18
Revision 1009 19
Revision 1009 20
Revision 1009 21
Revision 1009 22
Revision 1009 23
In this section we will look at the FIP creation of an Inter-switch Link between two FCFs. Revision 1009 24
This new example describes the process of an FCF identifying other FCFs and bringing up a virtual ISL between them. ELP is exchange link parameters. Revision 1009 25
In this multi-slide example, we have 3 FCFs. In this example, FCF C is generating the Solicitation. FCF C has just come online and will attempt to connect to a Fabric if a FC Fabric is available to connect to. FCF C sends out a multicast solicitation frame which is addressed to All-FCF-MACs (as shown in the bottom center of the diagram). The SA is the MAC address for itself, FCF-MAC(C). The Operation is a Discovery, Solicitation. The Descriptor list contains information indicating that the frame came from an FCF and which addressing modes are supported. Note that any time an FCF talks to another FCF, it will always use SPMA mode, which is a fixed MAC address, the MAC address that it started out with. There is no concept of changing the MAC address to match up with the FC_ID because E_Ports on switches do not get assigned 24-bit addresses. The other Descriptors include FCF_MAC(C), FC_MAP, SwitchName, and the Max FCoE Size. Revision 1009 26
After FCF A and B receive and process the multicast Solicitation frame, they will reply with unicast Advertisement frame, as shown in the bottom of the above diagram. This response frame carries all the necessary information for FCF C to bring up a link with each FCF. Additionally, note that these response unicast frames are create as mini-jumbo frame size. This is done to validate that the path supports mini-jumbo frames. If the path does not support mini-jumbo frames, then the frame will not arrive at FCF C, and the path will not be created. Revision 1009 27
FCF C will now generate a FIP Frame containing the ELP to transmit to FCFs A and B. At this point the traditional FC link initialization process will continue. Revision 1009 28
The ELP Response from A and B will complete the link set up. Revision 1009 29
At this point standard FCoE traffic can flow across the network links. Revision 1009 30
Revision 1009 31
Revision 1009 32
Revision 1009 33
Revision 1009 34