Fibre Channel E_Port Compatibility for IP Storage Networks

Transcription

1 Multi-Capable Network Solutions Fibre Channel Compatibility for IP Networks INTRODUCTION As one of the first applications for storage networking based on TCP/IP, extending connectivity for Fibre Channel SANs over long distances is an essential component of disaster recovery, remote backup and other critical business solutions. Since IP networks have no inherent distance or speed limitations, moving storage data over TCP/IP enables a more flexible and robust alternative to direct Fibre Channel-to-Fibre Channel linkage. For a number of reasons, native Fibre Channel has not proven viable for long haul applications between multiple sites. Customers who have attempted to extend Fibre Channel SANs have uncovered several issues: First, Fibre Channel was originally designed for high performance channel applications within the circumference of a data center. Fibre Channel physical standards allow for up to 500 meters using multimode fiber cabling, and up to 10 kilometers using single-mode cabling. The original design of Fibre Channel did not accommodate distances extending hundreds or thousands of kilometers, which might be typical of disaster recovery applications. Remote connectivity for Fibre Channel SANs is therefore inherently problematic since it pushes the architecture beyond its original purpose.

2 Second, this inherent distance limitation is affirmed by the amount of port buffers normally allocated by Fibre Channel fabric switches. Although the number of port buffers may be sufficient to sustain high bandwidth utilization within a local or metropolitan environment, current Fibre Channel switches lack the port buffering required for wide area links. As a consequence, performance drops dramatically when native Fibre Channel is driven beyond metropolitan distances. Third, connecting Fibre Channel switches over long haul links may expose the extended fabric to disruption if the wide area connections are over-subscribed or unstable. By design, a network built with two or more Fibre Channel switches becomes a single fabric. The expansion port () connection between two switches is used to establish a consistent addressing scheme, exchange of routing information and name service information for discovery and monitoring of devices. When an connection is broken, the single fabric may once again subdivide into isolated SANs. While this rarely occurs in a local data center environment, stretching connections over long distance links potentially exposes multi-switch fabrics to disruptions. Ideally, it should be possible to connect remote SANs together without the difficulties imposed by Fibre Channel distance limitations, inadequate port buffering and the internal mechanisms of Fibre Channel fabrics. Using native IP storage protocols resolves the issues of distance. Deep port buffering on an IP storage switch addresses the issue of performance of long distances. And limiting connectivity to each local SAN while simultaneously providing storage communications across wide areas eliminates the potential for fabric disruption. This is the solution offered by the McDATA IPS 3300 and 4300 storage switches. WHAT E_PORT CONNECTIVITY PROVIDES Expansion ports serve two basic functions for Fibre Channel SANs. The most common implementation for connectivity is to support higher populations of storage devices. A customer wishing to connect, for example, 100 Fibre Channel hosts and storage targets in a SAN would either have to purchase a large, 128 port director-class fabric switch or cascade multiple 16 port fabric switches together via s. The director switch option is more efficient, but comes at a significant cost. The 16-port switch option is more economical, but may create bottlenecks for storage traffic and impose lengthy convergence times to stabilize the SAN after initialization. As shown in Figure 1, seven 16-port Fibre Channel switches would be required to build a SAN supporting 100 end devices (6X15 usable ports + 10 usable ports for the center switch). In this example, s are used to connect the fabric switches together in a single SAN. Since only one is used for individual switch-to-switch connection, a potential bottleneck occurs each time multiple devices on one switch attempt to communicate with devices on downstream switch ports. This could be addressed by creating a meshed configuration between the switches with additional s, but that in turn would require more Fibre Channel switches to be added to sustain the 100 usable device ports. Using 16 port Fibre Channel switches, it is not possible to build a non-blocking multi-switch infrastructure for large SANs. Consequently, although connectivity enables higher populations of Fibre Channel devices in a single configuration, performance may suffer as the device count goes higher. Figure 1 connectivity is also used to simply tie together fabric switches over some distance. A fabric switch in an Engineering department, for example, could be connected to a fabric switch in a 2

3 storage resources. So while local connections may be very reliable within a site, the vulnerability of extended s to Tape Backup disruption have made native Fibre Channel SAN extension less attractive for customers who require stable operation for mission-critical applications such as remote mirroring, remote backup and disaster recovery. Data Center Engineering Figure 2 data center several floors away to provide consolidated tape backup. As shown in Figure 2, such connectivity may not be intended for sharing of all storage resources, but specifically for sharing of a targeted resource such as a tape library. In addition to switch-to-switch connection, then, it is desirable for the switches to exchange zoning information so that only authorized resources are made available via the connection. In this example, the two Fibre Channel switches in the data center are connected, forming a single fabric. When the Engineering Fibre Channel switch is also introduced, all three switches participate in a single Fibre Channel SAN. Similar to local connections, the failure of the DWDM or IP tunnel link would cause each site to devolve into separate SAN islands. In addition, without robust fabric switch design, instability at one of the sites may affect communications between both locations and intermittent loss of contact between servers and E_PORT STANDARDIZATION connectivity involves multiple Fibre Channel standards that define various processes that are performed between fabric switches. The standard most relevant to connectivity is NCITS/ANSI FC-SW-2. This standard was the product of a lengthy and often contentious effort by Fibre Channel vendors to agree on a common routing protocol known as Fabric Shortest Path First (FSPF). FSPF is a subset of the more commonly used and more robust Open Shortest Path First (OSPF) standard used in IP routing. Like OSPF, FSPF is a link state routing protocol, i.e. the best route through a network is determined by the optimum links (least cost) offered between switches. In the case of FSPF, the relative cost of traversing multiple switches is driven by the link speed provided by connections. For OSPF, additional variables such as current utilization or load may be factored into the cost equation. The FSPF routing protocol was not so much the result of technical collaboration between multiple vendors as it was a technical competition to force interoperability. It was only after a competing switch vendor reverse engineered a leading vendor s FSPF implementation with the intention of submitting it to open systems development in ANSI that the leading switch vendor finally relented and made initial efforts to interoperate with other switch vendors. Even then it took several years and industry pressure for interoperability to become sufficiently stable for customer 3

4 deployment. Currently, Fibre Channel switch vendors may offer several modes of connection, including a proprietary mode used by a leading switch vendor. The purpose of maintaining a proprietary mode, of course, is to lock customers into a single vendor solution by making interoperability problematic. Optimum routes through FSPF Fibre Channel fabrics are currently based on a single metric expressed in the formula: FSPF Cost = S(1000) X [Gbps] / Line Rate [Gbps]. Consequently, the cost of a standard 1 Gbps connection between two switches is As shown in Figure 4, the more links a connection must traverse, the higher the cost. Since the path from switch A to switch D has least cost when routed via switch B, the more costly path through C will not be selected unless the link between switch B and switch D fails. The cost calculation becomes more interesting (and less like simple Routing Information Protocol or RIP) when mixed 1Gbps and 2 Gbps connections are deployed. A proposal for a version 2 of FSPF (FSPFv2) would introduce the variable of latency into the link cost equation. This initiative has been prompted by initial experience with native Fibre Channel SAN extension over longer distances in which speed of light latency impacts performance. By factoring in latency, it is possible to select a more optimal route that provides lower bandwidth but less latency over a higher speed link with more latency. In addition to routing information, it is also desirable to exchange zoning information over connections. The ability to merge zoning tables between Fibre Channel switches allows administrators to allocate storage resources to the appropriate hosts even as SANs scale to larger multi-switch configurations. How zoning is implemented in individual switches may be proprietary, but establishing a standardsbased means to share zone information between different switch products facilitates deployment of more complex SANs. Tape Backup Site A Extension DWDM or Tunnel Figure 3 DWDM or Tunnel Site B The exchange of FSPF routing and zoning parameters over connections helps build the framework of the fabric. The fabric itself, however, is only a convenience for communication between servers, disks and tape devices. Another essential ingredient for connectivity is therefore exchange of Simple Name (SNS) data so that servers can discover and establish connections with storage resources throughout the fabric. The name server standard is covered by NCITS/ANSI FC-GS-2 and for multi-switch configurations relies on compliance with FC-SW-2 for successful exchange of SNS information. When combining connectivity with native IP storage protocols, the SNS mechanism is complemented by the Internet Name Service (isns) to enable initiators such as servers to discover both local and remote SAN resources. To avoid duplication of addressing between multiple Fibre Channel switches, Fibre Channel connectivity requires a process for 4

5 established and then lost. While this is not likely to occur in a A 1000 B controlled data center environment, metropolitan and wide area extensions may be vulnerable to disruption. Link Cost A to D: 2000 A to D via C: 3000 D Figure 4 assigning blocks of unique addresses to individual switches. In the Principal selection process, one fabric switch assumes the role of primary switch and automatically allocates blocks of addresses to the other switches in the fabric. In a single switch SAN, an individual switch would serve as its own Principal. The Principal facility was developed to automate link layer addressing in Fibre Channel fabrics. In practice, most vendor implementations allow (or require) an administrator to manually determine which switch will serve as primary and which as secondary. Two stand-alone switches that are then connected by to each other will initiate a Principal selection process and block address assignment, as well as reconfiguration of addressing for any attached storage devices and hosts. Zone merging between the combined switches will, depending on vendor implementation, allow discovery of authorized end device by Worldwide Name (WWN) or port attachment. s may query the SNS database to discover, in addition to previously found local resources, the new storage resources attached to the partner switch. If, subsequent to fabric creation, the connection is broken, the fabric may split again into two separate SAN islands. In native Fibre Channel implementations, there is no means to maintain the autonomy of each SAN island and yet simultaneously allow communication between islands. es are either wholly separate, or wholly combined into a single storage network. This all or nothing approach to fabric building may result in a disruptive reconfiguration cycle as connections are C MCDATA FIBRE CHANNEL E_PORT SOLUTIONS To overcome the inherent limitations of distance, buffering and fabric reconvergence specific to Fibre Channel fabric switches, McDATA IP storage switches have been designed to supply both compliance and enhanced IP services more suitable for metropolitan and wide area storage applications. These enhanced services include IP storage protocols (iscsi and ifcp), full layer 3 IP routing, OSPF support, quality of service (QoS) and compatibility with IP security (IPSec) solutions. In addition, McDATA IP storage switches provide enormous buffers (256MB) for long haul applications, enabling full bandwidth utilization at gigabit speeds over thousands of miles. As shown in Figure 5, McDATA IP storage switches are compliant for attachment to local Fibre Channel switches. In addition to connection to existing Fibre Channel switches, however, the McDATA IP storage switches also support direct attachment of Fibre Channel end devices (as shown at Site B). With attachment to a Brocade, McData or other Fibre Channel switch, the McDATA IP storage switch appears as another Fibre Channel switch in the local fabric. At Site B in the diagram, for example, the local Fibre Channel switch and McDATA IP storage switch would undergo Principal selection, assign address blocks, merge zoning information, exchange FSPF routing tables and SNS information per Fibre Channel standards. The server attached to the McDATA IP storage switch would, upon querying the SNS, discover the storage array attached to the Fibre Channel switch at Site B, just as if it were attached to any standard Fibre Channel fabric. What distinguishes this configuration from conventional Fibre Channel fabric extension, however, is that all the appropriately zoned devices at Site A would also appear as local resources at Site B. Unlike native Fibre Channel extension via DWDM or IP tunneling, there is no need to stretch an connection across the IP cloud 5

6 between the IP storage switches. Instead, each IP storage switch at each site appears as a large virtual Fibre Channel switch to its local fabric, supporting a multitude of local devices. In reality, those devices could be dispersed anywhere across the IP routed network. This edge switch strategy accomplishes two critical functions. First, by being fully Tape Backup compliant, the McDATA IP storage switches can bring entire Fibre Channel fabrics into an IP-based SAN. This allows the McDATA IP storage Site A switches to be deployed in place of (for new installations) or complementary to (for existing installations) Fibre Channel switches. Second, by maintaining the autonomy of each Fibre Channel SAN island, the McDATA solution avoids the vulnerabilities of native Fibre Channel SAN extension. Each local SAN maintains its integrity and is not forced into a reconvergence if difficulties occur on the wide area links. Maintaining local autonomy while providing storage communications across the IP network leverages the best of both Fibre Channel and IP internetworking. McDATA compliance also facilitates scalability. Any number of remote sites can be joined via the IP storage network with any mix of Fibre Channel fabric switches or Fibre Channel end devices. As shown in Figure 6, the use of standard OSPF routing and native IP storage protocols enables scaling to larger IP SANs. From the IP Network IP Figure 5 IP IP Site B standpoint of the initiator at Site A, for example, the introduction of Site C and Site D simply adds more local storage devices to the local fabric created by the Fibre Channel and McDATA IP storage switches at Site A. As with any Fibre Channel switch, the McDATA IP storage switch sends the proper state change notification that, depending on zoning parameters, alert servers that new storage resources are now available. Contrary to normal Fibre Channel behavior, if a remote site fails, no reconvergence of the SAN is required. The McDATA IP storage switches simply notify the appropriate devices that the affected storage resources are no longer available. Accommodation of Fibre Channel switches through connectivity also facilitates migration from Fibre Channel to native iscsi SANs. Since McDATA IP storage switches support both Fibre 6

7 Channel-to-Fibre Channel and iscsi-to-fibre Channel communications, customers can also deploy iscsi storage hosts and targets over time. This provides flexibility in selecting the best products available in the market while preserving the customer's investment in SAN solutions they have already acquired. SUMMARY While Fibre Channel connectivity is useful for cascading Fibre Channel switches, multi-switch Fibre Channel SANs have unique requirements. Fibre Channel was not originally intended for wide area SAN applications. Consequently, extending native Fibre Channel s over long distances may expose stretched fabrics to disruption. An optimum solution preserves interoperability at each local SAN while using native IP storage protocols to span wide area links. The McDATA IP storage switches provide both compliance for connection to existing Fibre Channel switches and a more robust and stable native IP solution for metropolitan and long haul connectivity. By preserving the autonomy of each local SAN, SANwide disruptions and reconvergence can be avoided. Support of zone merging insures that only authorized devices can communicate across the IP storage network. Site D Site C IP IP IP Network IP Tape Backup IP Site A Site B Figure 6 7

8 The combination of compliance and support for iscsi allows customers to integrate the best of class Fibre Channel and IP storage into a coherent storage network. By not forcing customers into architecture, protocol or infrastructure decisions, customers can select the appropriate storage solutions and maximize their storage investment. Site D Site C IP iscsi iscsi s IP IP IP Network iscsi s Tape Backup iscsi IP Site A Site B Figure McDATA Corporation. All rights reserved. McDATA, the McDATA logo, Blended Fabric, Fast Write, Fabricenter, HotCAT, Intrepid, Multi-capable storage network solutions, Networking the World's Business Data, OPENready, SANavigator, SANpilot, SANtegrity and Sphereon are trademarks or registered trademarks of McDATA Corporation, SANavigator, Inc. or Nishan Systems, Inc. OEM and Reseller logos are the property of such parties and are reprinted with limited use permission. All other trademarks are the property of their respective companies. All specifications subject to change. MC