SONET Links Extend Fibre Channel SANs Research Brief Abstract: Emerging product that support the transport of storage commands and data over Synchronous Optical Network or Synchronous Digital Hierarchy are extending Fibre Channel storage-area networks over WAN or metropolitan-area network infrastructures. By James Opfer Recommendations SAN component vendors should expedite full interoperability with Fibre Channel (FC) over Synchronous Optical Network (SONET) or Synchronous Digital Hierarchy (SDH) storage-area network (SAN) extension devices. Providers of business continuance infrastructure should carefully assess the optimum link to the WAN or metropolitan-area network (MAN) infrastructure. Vendors should provide applications supporting business continuance that accommodate delays associated with long-distance SAN extension. Publication Date: 7 February 2003
2 SONET Links Extend Fibre Channel SANs FC Over SONET Emerges as a SAN Extension Technology The dominance of FC technology in SANs and the widespread availability of SONET infrastructure provide an opportunity for flexible, economical transport of block data over long distances. Current market conditions favor this technology, and FC over SONET products already deployed have gotten enthusiastic endorsement. Supported solutions include disk mirroring, tape backup and file sharing all over WAN-like distances. FC SAN Extension: A Critical Technology Supporting Business Continuance Effective implementation of business continuance and disaster recovery processes requires rapid duplication of data at sites separated by appropriate distances. These distances vary with the extent of possible disruptions and disasters that might otherwise interrupt business activity. The "Draft Interagency White Paper on Sound Practices to Strengthen the Resilience of the U. S. Financial System" (Securities and Exchange Commission Release No. 34-46432, File No. S7-32-02, September 2002) suggested 200 to 300 miles between primary and backup sites as a starting point for discussion. In response, numerous comments from financial institutions and technology providers detailed the distance limitations of current technologies. While the U.S. government has temporarily backed off from requiring the suggested separation, the original suggestion reflects real needs for adequate separation to protect against various disasters on a metropolitan scale. Such real needs add to the existing demand for more effective, economical technologies to support business continuance. While point-to-point connection of primary and secondary systems can support backup and recovery, the alternative architecture of SANs provide greater flexibility. Remote servers and storage systems within the reach of the SAN can exchange data under the control of suitable management software. Hence, detailed SAN attributes markedly influence deployment of business continuance solutions. Although Internet SCSI (iscsi) is emerging, FC will remain the technology of choice for transferring blocks of data rapidly on a SAN for at least the next three years. Users typically deploy FC SANs as islands on the scale of a data center or campus with links that connect these islands to similar islands. FC supports interconnection on a campus scale with copper connections for short distances, multimode optical fiber for up to 500m and single-mode optical fiber for greater distances (typically 10 kilometers [km] in 2002). Current 2 gigabits per second (Gbps) FC SAN technology supporting data transfer rates of 200 megabytes per second (MB/sec) provides a complete connectivity solution for business continuance on the scale of a few kilometers, as long as it is practical to provide cable dedicated to FC. As SANs become the dominant technology for connecting servers and storage systems over distance, extending their range provides the infrastructure to support rapid duplication of data at widely separated
sites. Connecting FC SAN islands to extend the range of SANs beyond the reach of pure FC requires technologies that complement standard FC. Technologies implemented for FC SAN extension divide into two categories. In the first category, FC extenders classified here as "transparent" facilitate transmission of the FC signal over greater distances. Conventional FC flow control is utilized with increased buffering to enable a larger number of FC frames in flight. Dense wave division multiplexing (DWDM) optical transport platforms primarily comprise this category, which also includes special extended distance gigabit interface converters (GBICs). Deployment of DWDM technology, enabling transmission of multiple channels over one single-mode optical fiber, typically occurs in MANs. The limit for the range of these transparent extensions is about 100 km. In the second technology category, FC frames carried by the FC signal are encapsulated in another protocol at one of two extension devices, transported over distances that may be characteristic of WANs and reconstituted as an FC signal by the second device. FC flow control is typically implemented only up to the extension device. Between extension devices at the ends of the link, data rate and flow control are determined by the extension devices and the alternative protocol. Just as manufacturers have given various names routers, gateways, service platforms and others to the extension devices, there is a multiplicity of the highest-level protocols utilized for encapsulation. Asynchronous transfer mode (ATM) was the first widely employed protocol, followed by a wave of interest in Ethernet and Internet Protocol (IP). Supported by this interest, the specification for encapsulating FC over IP (FCIP) is well along the path to becoming an Internet Engineering Task Force (IETF) standard. While it was natural for the first FC extension technologies to utilize available protocols with full network capabilities, the extension functionality does not require this full network capability. This is where SDH and its North American equivalent, SONET, enter the picture. Because SONET/SDH provide much of the lower-level transport for ATM and IP, the logical extension of the argument is that the SONET alone can provide the encapsulation and transport required for FC SAN extension. Indeed, Akara in 2001 and Lightsand Communications in 2002 introduced products enabling FC SAN extension over SONET/SDH infrastructures, denoted simply as FC over SONET. The attractive attributes of this emerging architecture merit further examination of its position relative to alternatives. 3 Business Trend Because of the advantages of connecting servers and storage devices by means of SANs, Gartner Dataquest forecasts that, by 2006, the market share for SAN-attached external redundant array of inexpensive disks (RAID) controller-based storage will have grown to 56 percent of the $24 billion RAID storage market. For more information, please see the Gartner Dataquest Market Statistics report " Disk Storage Systems Worldwide Forecast: 2002-2006" (HWST-WW-MS-0126). In anticipation of this trend,
4 SONET Links Extend Fibre Channel SANs as early as 1999, several vendors were developing encapsulation technologies for SAN extension with increasing interest in Gigabit Ethernet and IP as the transport protocol. In 2000, Computer Network Technolgy (CNT), a supplier of storage networking solutions over ATM since 1995, successfully introduced FC extensions over IP to complement its ATM products. FCIP-based products exploit the connectivity of IP networks but deliver predictable performance only if used on a network with performance tailored to storage requirements. As these SAN extension products extend to WAN dimensions, they almost inevitably involve SONET links along the way. IP storage networking technology, which includes FCIP, is not yet as ubiquitous as was anticipated, and FC has continued to fortify its dominant market position. In this market environment, FC over SONET extension products that eliminate the intermediate ATM and IP protocols emerged quite naturally. Coincident with this emergence is the supporting work in standards bodies. Examples include work on the Generic Framing Procedure (GFP) in the T1X1.5 Technical subcommittee of the Alliance for Telecommunications Industry Solutions and work on the FC Backbone Standard (FC-BB-2) in the T11.3 Group of the International Committee for Information Technology Standards. In addition to a favorable cost structure of enabling platforms, FC over SONET also supports the trend toward lower costs driven by greater utilization of optical bandwidth. Products such as the Cisco ONS 15530 Metro DWDM platform promise to aggregate multiple higher-speed networking and storage services, such as Gigabit Ethernet and FC, over individual wavelengths in the future. FC over SONET across DWDM already provides many of the economic benefits of time multiplexing superimposed on wavelength multiplexing. Gartner Dataquest Perspective Current reality favors rapid implementation of FC over SONET. As noted earlier, SONET is the common transport supporting data traffic over distances extending to thousands of kilometers. FC, in turn, is the dominant technology for transport of block data over networks on a campus scale. While iscsi and associated IP storage technologies are emerging, they are not necessary as an intermediary for a point-to-point connection between two widely separated sites. The full networking capabilities of IP storage are important only if there is a complex distribution of nodes requiring the interchange of data. FC over SONET requires only validation of interoperability as a SAN extender and a connecting link between the FC over SONET platform and the nearest available SONET ring. A strong endorsement of the validity of this concept already appears on an IBM Global Solutions Directory Web page describing the Akara platform used to enable disk mirroring. The Akara product is described as "supporting much lower cost points than currently available for disaster and business continuance solutions." Where suitable SONET connections are provided, FC over SONET is a real solution available now. In further support of this position, on its Web site, Silicon
5 Graphics provides a strong endorsement of Lightsand's technology as an enabler of shared file systems over WAN distances. Perhaps the most complicated issue related to the utilization of FC over SONET is the accessibility of the SONET infrastructure to the enterprise hosting the FC SAN. Sometimes referred to as the "first mile" problem, it engenders active competition between native SONET and Gigabit Ethernet. Clearly, if the first mile deployment is Gigabit Ethernet even if over native SONET, it would favor IP-based SAN extension, such as FCIP. If the first mile is just native SONET, then FC over SONET for SAN extension is highly compelling. Since the first mile problem varies dramatically with location, it is difficult to characterize generally. However, this part of the problem demands special attention in any plans for business continuance infrastructure. Because FC SANs move data between ports at 1 or 2 Gbps and FC over SONET products support these data rates, the connection to SONET rings ideally would also do so. This suggests OC-48 (2.488 Gbps) for the SONET connection to match FC SAN performance. In reality, today's disk mirroring and tape backup applications exercise only a portion of the potential throughput of FC links. Consequently, lower speed versions of SONET are likely to be a suitable match. These lower-speed SONET services are more readily available and less expensive. The wide range of performance available and the matching wide range of costs become the critical parameters in the planning process, and it must be so recognized. Vendors of application and SAN management software should find significant opportunities for software that optimizes the application of these high bandwidth SAN extension links. Whatever the alternatives, FC over SONET provides nearly all of the ideal characteristics of a longdistance link for FC SAN extension. In fact, the residual issue of irreducible latency can be addressed as though the link has ideal performance. Signals propagating over typical optical fiber experience 1 millisecond of delay for each 200 km, not too different from the physical limit of 1 millisecond for each 300-km characteristic of light propagation in vacuum. The 50-millisecond round-trip delay for a 5,000-km transcontinental link is well outside the acceptable limits for an unmodified application designed for storage on a local- or even metropolitan-scale SAN. Software must deal explicitly with the latency parameters. SAN management software should provide visibility of the special properties of the high-latency SAN links. Providers of software for the management of business continuance obviously must provide solutions with parameters that optimize performance for whatever link delay is present. While specialty solutions are presumably available today, the real opportunity is to provide these solutions in an automated fashion. With FC over SONET and appropriate software, applying SAN technology to business continuance on a continental scale becomes an increasingly attractive reality.
6 SONET Links Extend Fibre Channel SANs Key Issue What changes in technologies and vendor dynamics will shape the storage industry? This document has been published to the following Marketplace codes: HARD-WW-DP-0421 For More Information... In North America and Latin America: +1-203-316-1111 In Europe, the Middle East and Africa: +44-1784-268819 In Asia/Pacific: +61-7-3405-2582 In Japan: +81-3-3481-3670 Worldwide via gartner.com: www.gartner.com Entire contents 2003 Gartner, Inc. and/or its Affiliates. All rights reserved. Reproduction of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice. 112958