Resilient Packet Ring Technology Overview Technical Note
Table of Contents Introduction 1 Today s Traditional Metro/Access Networks 1 RPR Combines the Best of Both Worlds 3 SONET Over RPR 8 Summary 9
Introduction Resilient Packet Ring (RPR) is a new technology being standardized in the IEEE (802.17 Workgroup). The objective is to enable a true alternative to SONET transport for packet networks, providing carriers with resiliency, fast protection and restoration, and performance monitoring comparable to those of SONET networks. This Technical Note provides a general overview of RPR technology. It is focused on a single application of RPR technology, which we consider as the most important one - building metro core and access transport networks. We begin by discussing today s solutions to metro core and access networks, including their strengths and limitations. We continue by describing RPR s main benefi ts and move to describe in some more detail some of its technological aspects. Today s Traditional Metro Core and Access Networks Solutions for metro core and access networks have been proposed by numerous vendors using various proprietary and standards-based technologies. All of these solutions have tried to address two key problems: 1) the ambiguity of supporting existing legacy traffi c and services, while migrating to a new future-proof dataoptimized infrastructure and 2) the disconnect between the growth and profi tability of services. Legacy services are predominantly TDM -based, while data services to end-users are carried by Ethernet, Packet over SONET, ATM, and Frame relay technologies. Operators are looking to combine legacy and future-proof services over a converged multiservice network that will carry data, voice and video traffi c over the same infrastructure, thus reducing both equipment and operational costs. Multiservice networks are not, however, a new concept. In past years ATM promised to be a unifying technology for public networks. Vendors developed a broad range of multiservice ATM switches for both the core and edge of networks. However, the drawbacks of ATM became apparent when bandwidth demands increased exponentially; ATM could not scale for several years beyond the OC-12 level nor cost effectively provide TDM, or IP services. These ineffi ciencies, in the face of the explosive growth carriers faced, has led us to the consensus that IP will ultimately be the true convergence layer for both real-time and non-real time applications. Ethernet, and more specifi cally Gigabit-Ethernet, is expected to be the interface of choice for enabling this convergence due to its low cost, native LAN connectivity, synergism with IP, bandwidth granularity and future bandwidth scalability (10GbE). Most of today s metro/access networks consist of different network elements for transport (typically SONET ADMs), layer-2 aggregation and switching (ATM, Frame Relay or Ethernet) and layer-3 IP routing. Recently, two major alternative approaches have been introduced to try and eliminate this dependence on multi-ne architectures - Next Generation SONET Solutions, and Ethernet-based Metro Switches. Next-generation SONET solutions look to integrate data functionalities into a common SONET layer 1 solution, while Ethernet transport solutions migrate today s enterprise Ethernet switches to the metro network, attempting to replace SONET as the layer 1 technology, while establishing Ethernet as the technology for the aggregation and switching layer. Next Generation SONET Solutions Next generation SONET solutions offer integration of several functionalities into a single network element, reducing Page 1 of 9
the power and space requirements as well as facilitating the confi guration and management of the network. Next generation SONET solutions integrate more data handling capabilities into the add-drop, and distributed crossconnect, functionalities associated with traditional SONET. The goal is primarily one of improving the bandwidth granularity beyond the traditional STS-Nc boundaries, while maintaining all of the carrier-class characteristics of SONET, such as fast protection and restoration and performance monitoring capabilities. Next generation SONET solutions are addressing carriers need to continuously support their high revenue TDM services, while enabling them to offer new Ethernet-based data services. TDM services are not going to disappear in the near future: On the contrary, they are experiencing a tremendous growth as the primary means of carrying today s data traffi c. According to RHK, revenues from IP services will grow to $25B by 2003, while non-ip services, including Frame Relay, ATM and TDM private line will account for $50B in the same year. Non-IP services represent today the vast majority of carriers revenues from data services; Carrier s carriers often need to support their customers existing installed base, which is mostly based on TDM and SONET. For example, a carrier offering wholesale transport services to another carrier operating a last-mile broadband network will have to support DS-3 and OC-3 interfaces, since today s installed base of DSLAMs and last-mile wireless hubs is connected to the metro network by these interfaces. Furthermore, OC-3 and OC-12 interfaces are commonly used today to interconnect ISPs ATM switches and routers. However, despite their extensive support for TDM services, next-generation SONET solutions do not offer a truly data-optimized solution to the carrier. Bandwidth utilization is relatively low over SONET networks; 50% of the potentially available bandwidth is reserved for protection, while statistical multiplexing, when implemented (which is rare), is done on a node level, where a better statistical multiplexing gain can be achieved by multiplexing on a ring level. Furthermore, the cost curve of next-generation SONET solutions, although improving space and power requirements, is not dramatically different from traditional SONET equipment. Ethernet Solutions Ethernet technology is well suited for the data centric needs of the future network edge. After all, the source of the majority of this data traffi c is from high-speed LANs that are overwhelmingly based on Ethernet. Ethernetbased systems also scale well from 10Mbps to 1Gbps today with 10Gbps on the horizon. These characteristics make Ethernet an extremely attractive alternative to service providers (CLECs, ICPs, etc.) with no current installed base, deploying data-only networks, and offering Ethernet-based services to other service providers and enterprises. However, this poses two probems: since Ethernet was designed for the enterprise market, it does not provide the necessary service quality of telco carrier-class systems that must support 99.999% service availability, including functionalities such as service restoration within 50msec in the event of a fi ber cut or node failure; nor does it address the need to support the huge demand by customers for legacy services. Ethernet based solutions were conceived for mesh based network architectures, however most incumbent carriers are building ring-based metro core and access networks. This is due to the clear advantages of a ring topology over a star topology when protection is required (almost half the fi ber mile and long-haul laser counts) Ethernet suffers severe fairness and protection problems when operating on ring architectures; The fairness problem results from the fact that Ethernet switches provide the same priority for every interface on an Ethernet switch. Page 2 of 9
If all the traffi c on a ring is destined to a hub node, the node closest to the hub will get half of the bandwidth, the next node will get a quarter etc., creating unfairness between users with the same service level agreement, but with different distances to the destination node. As for protection, in a mesh topology, protection of Ethernet switches can be done in the order of 1 second, while it would take tens of seconds to achieve restoration in a ring topology. This is attributed to the restoration mechanism that is built on a distributed topology discovery protocol (known as the Spanning Tree Protocol) that has to robustly converge before making a rerouting decision in order to prevent loops (which have a severe effect on an Ethernet network, up to a need to reset all the network). There are recent attempts to speed up the Spanning Tree Protocol, but they cannot achieve the 50ms restoration time criterion. A further important challenge to Ethernet transport, already mentioned above, is the inability to effi ciently transport high-revenue TDM, Packet over SONET, ATM or Frame Relay services over it. ILECs, IXCs and CLECs expect to offer these high margin services for many years and investing in another new network that cannot carry these high-revenue services is viewed as unattractive. Taking the above considerations into account, Ethernet could be successfully deployed for data-only, green-fi eld deployments that do not interface to legacy networks and services. However, operators that require carrier-class functionalities such as fast protection and restoration and fairness, as well as the ability to support profi table TDM, Packet over SONET, ATM or FR services, will probably be reluctant to adopt Ethernet as a metro core or access transport technology. RPR Combines the Best of Both Worlds Resilient Packet Ring (RPR) technology was designed to combine SONET s carrier-class functionalities with Ethernet s high bandwidth utilization and granularity. Additionally, RPR technology offers fairness that has been lacking in today s Ethernet solutions. RPR is a new MAC layer technology, being standardized in the IEEE 802.17 workgroup. This employs spatial reuse to maximize bandwidth utilization, provides a distributed fairness algorithm, and ensures high-speed traffi c protection similar to SONET Automatic Protection Switching (APS). RPR allows full ring bandwidth to be utilized under normal conditions and protects traffi c in the case of a nodal failure or fi ber cut using a priority scheme, alleviating the need for SONET-based protection. Furthermore, because the RPR MAC layer can run on top of a SONET PHY, RPR-based networks can provide performance-monitoring features similar to those of SONET. RPR technology offers all of these carrier-class functionalities, while at the same time keeping Ethernet s advantages of low equipment cost, high bandwidth granularity, and statistical multiplexing capability. Technical aspects of RPR s spatial reuse, fairness, fast protection and quality of service capabilities are discussed in the following paragraphs. Spatial Reuse One of the main features of the RPR protocol is its spatial reuse capability. Spatial reuse is a concept used in ring topologies to increase the overall aggregate traffi c on the ring. This is possible by allowing traffi c to be passed Page 3 of 9
bi-directionally on the ring only on spans between source and destination nodes. The destination node strips its packets from the ring thereby freeing the full bandwidth on other ring segments for utilization by other packets. This is different from earlier ring based protocols, such as token ring and FDDI, where traffi c was removed from the ring by the source node, occupying unnecessary bandwidth. RPR is similar in this sense to SONET two-fi ber Bi-directional Line Switched Ring (BLSR), but unlike SONET BLSR, it allows full utilization of the ring bandwidth with non-preemptable traffi c. Figure 1 demonstrates how spatial reuse works. Assume the ring runs RPR on top of OC-192c, and there s a substantial amount of non-guaranteed traffi c over it. In this example node 6 is sending 2Gbps traffi c to node 1 and node 5 is sending 7Gbps traffi c to node 1. On a traditional SONET ring, this would fi ll up the ring capacity, while in an RPR ring, nodes 3 and 2 can continue to utilize the full ring bandwidth. Figure 1: Spatial Reuse in an RPR Ring Current analysis on metropolitan networks traffi c forecasts that over 80% of the metro traffi c is expected to remain inside the same metro network. Under these conditions spatial reuse mechanisms in metropolitan rings are extremely advantageous. The quantitative advantage of spatial reuse can be calculated by simulations, and is dependent on traffi c statistics, as well as on the number of nodes in the ring. A specifi c case study demonstrating the strengths of spatial reuse as well as statistical multiplexing capabilities in deployment of metro networks is shown in a separate Corrigent Application Note. Fairness Fairness is one of the most important features in carrier-class networks. Fairness is achieved when the traffi c characteristics of two service fl ows that have the same service level agreement (SLA) are identical, regardless of their network source and destination. The RPR protocol can guarantee fairness across the metropolitan network. Each node on the metro ring executes an algorithm designed to ensure that each node on the ring will get its fair share of bandwidth. However, ring nodes use the spatial reuse capability to use additional available bandwidth, that is greater than their fair share on local ring segments, as long as it doesn t affect other ring nodes. More specifi cally, the ring supports weighted fairness, proportional to the bandwidth each user buys. Page 4 of 9
Figure 2 demonstrates the operation of the fairness algorithm. Before employing the fairness algorithm, all nodes transmit at their peak usage rate. At some point Node 1 experiences congestion and requests upstream nodes to reduce their transmission rate to the allowed rate. After convergence, each node will receive its fair share of bandwidth, while nodes that can locally transmit higher rates, like nodes 2 and 3, without generating congestion on other nodes, continue to transmit their peak usage rate. Other data optimized technologies, such as Ethernet, do not provide the carrier-class fairness guaranteed in RPR-based networks. Ethernet switches prioritize traffi c locally, on every interface on the ring, thus creating unfair conditions for traffi c that has to traverse through several nodes on its way to the destination node. Figure 2: Fairness in an RPR Ring Figure 3 demonstrates the lack of fairness in Ethernet-based networks. In this example, a 10Gbps Ethernet ring connects between the nodes. Nodes 4, 5 and 6 all transmit traffi c to node 1, creating congestion between node 1 and node 6. Under these conditions, node 6 will get half of the total available bandwidth, while nodes 4 and 5 will each get 25% of the available bandwidth. Figure 3: Unfairness in an Ethernet Ring Page 5 of 9
Fast Protection and Restoration The feature that more than any other makes RPR a carrier-class technology is its fast self-healing capability which allows the ring to automatically recover from a fi ber cut or node failure. This is done by wrapping traffi c onto the alternate fi ber within the SONET-class 50msec restoration timeframe. RPR technology provides carriers with more than SONET-class fast protection and performance monitoring capabilities, it also enables them to achieve it without dedicating protection bandwidth, as in the case of SONET, thus eliminating the need to sacrifi ce 50% of the ring bandwidth for protection. Two mechanisms are proposed for fast protection in the RPR MAC - Wrap and Steer. Each of these mechanisms has its own advantages and limitations, and both can be used on an RPR ring utilizing the Selective Wrap Independent Steer (SWIS) scheme. More on SWIS in a separate Corrigent Technical Note. Figure 4 shows an example of the data paths taken before and after a fi ber cut event when Wrap is being used. Before the fi ber cut, node 3 sends traffi c to node 1 via node 2 (Figure 4a). When the fi ber cut occurs (between node 1 and node 2), node 1 will wrap the inner ring to the outer and node 2 will wrap the outer to the inner ring. After the wrap, traffi c from node 3 to node 1 initially traverses through the non-optimal path (passing through node 2, back to node 3 and into the originally assigned port on node 1 - Figure 4b). This is immediately done and can inherently meet the 50ms criteria, as it is a local decision of the nodes that identify the fault (in this case, nodes 1 and 2). Subsequently, higher layer protocols discover the new ring topology and a new optimal path is used (Figure 4c). This can take several seconds to converge in a robust manner, after which the double ring capacity is restored (even though not evenly distributed, depending on the location of the fault and the ring traffi c pattern). Thus, unlike SONET rings that always pay the redundancy tax to achieve protection, after a failure, an RPR ring reduces its capacity only for several seconds. Other data optimized protocols, such as Ethernet, do not provide the SONET-class restoration time. Typically, spanning tree tools are used for protection in these cases. In a mesh topology, these tools provide protection time of the order of 1 second, but in ring topologies, protection time is degraded to the order of 10 seconds or more. Figure 4: Fast-Protection in RPR Using Wrap Page 6 of 9
In summary, RPR s fast protection and restoration capability prevents service loss for high priority critical traffi c. And just as SONET does, RPR enables carriers to continue to support their existing critical traffi c over a data optimized network, without compromising their guaranteed 99.999% service availability. Quality of Service Quality of Service (QoS) is required in order to let a carrier effectively charge for the services it provides. ATM promised to deliver multiple services due to its rich QoS feature set. However, a carrier s service offering should be simple. Customers should clearly understand the service differentiators for which they re required to pay. Often, a too-rich QoS feature set causes a complicated and incomprehensible service offering. Furthermore, different service quality features are required for different types of applications. Data transfer applications require low packet loss rate, while real-time applications, such as voice, require low latency and low delay variation. There are several parameters that more than others govern the characteristics of a delivered service: Service availability, delay, delay variation and packet loss rate. Service availability depends on the reliability of the network equipment, as well as on the network survivability characteristics. Delay occurs in a network when a packet waits in a switch queue for other packets to pass. Delay variation is the difference in delay of several packets belonging to a common traffi c fl ow. High-frequency delay variation is called jitter while low-frequency delay variation is called wander. The most important parameter for a carrier selling bandwidth services is service availability. In order to remain competitive, a carrier must guarantee fi ve-nines (99.999%) service availability, irrespective of other service characteristics. The respective requirement from the network is that all traffi c fl ows should remain active under any circumstances, including during a protection event. During a protection event, and until high layer protocols converge, the available bandwidth on an OC-192 RPR ring reduces to 10Gbps. In order to guarantee service for every traffi c fl ow under these circumstances, a portion of each fl ow s bandwidth should be allocated on this guaranteed throughput. This way, even during a protection event, service availability is guaranteed and none of the services is preempted. Traffi c delay is a minor issue in high throughput rings. An end-to-end delay of several tens of milliseconds is usually regarded as acceptable, while in an OC-192 ring, the expected delay for the longest Ethernet frames (1518 bytes) is shorter by three orders of magnitude (about 5(sec). Delay variation control is essential to ensure proper transport of TDM services over an RPR ring. Control can be achieved by synchronizing the nodes on the RPR ring by a common timing source. Policing is another tool that lets a carrier differentiate between service classes. In an underutilized network that does not employ policing, customers may get a better service than they ve paid for. In this case, they ll be reluctant to upgrade the service they re buying, thus reducing the carrier s revenue potential. A per-fl ow policing ensures that customers get only the service they ve paid for. Page 7 of 9
SONET over RPR RPR technology is designed to enable service providers to build carrier-class data optimized networks. However, most of the revenues of carriers today come from TDM services over their legacy TDM network. The ability to provide TDM and SONET services over RPR-based networks enables carriers to keep their current high-revenue SONET-based services, as well as offer new services over their packet optimized network and gradually migrate to Ethernet-based services. Thus, delivering TDM/SONET services over RPR-based networks is an essential complementary capability to the RPR technology. Since TDM/SONET services are delay and delay variation sensitive, delivery of these services requires strict jitter and wander control as well as end-to-end synchronization over the ring. Standardization of SONET services over packet networks has been progressing in the IETF s pwe3 workgroup. Its goal is to enable endto-end delivery of L1/L2 services over metropolitan as well as long-haul packet networks. Currently, standardization is focused on circuit emulation services over Multi-Protocol Label Switching Protocol (MPLS). MPLS labels and a new circuit emulation header are used to encapsulate SONET frames and provide the Circuit Emulation Service over MPLS (CEM). Essentially, a SONET stream is segmented into packets and encapsulated in MPLS packets. Each packet has one or more MPLS labels, followed by a CEM header to associate the packet with the SONET stream. The outside label is used to identify the MPLS Label Switch Path (LSP) used to tunnel the TDM packets through the MPLS network, whereas the interior label is used to multiplex multiple SONET connections within the same tunnel. Until circuit emulation of SONET over MPLS is fully standardized and interoperable, aggregation of the SONET-based traffi c headed outside of the metro ring can be done in a central point, and handed over to a long-haul legacy network. This way, a service provider can continue to support TDM/SONET services of its users, without operating a legacy network in the metro core or access area. An RPR-based network, supporting transport of TDM/SONET services, can aggregate using a Virtual Digital Cross-Connect, DS-1, DS-3, OC-3, OC-12 and OC-48 services into multiple OC-48s or OC-12s in the carrier s PoP, and hand them over to a long-haul SONET network. This eliminates the need to deploy a legacy network, including cross-connects and add-drop multiplexers, in the metro core oraccess network. Figure 5: SONET Services Page 8 of 9
Figure 5 demonstrates how SONET services are supported over RPR-based networks. In Figure 5, nodes 2, 3, 4 and 5 serve users with DS-3, OC-3 and OC-12 services. In this example the DS-3 services are connecting users within the same ring (between nodes 2 and 5) and the traffi c is carried transparently over the RPR ring, while the OC-3 and OC-12 services, connecting users in other metro rings, are aggregated into a single OC-48 port in the carrier s PoP (node 1), and handed over to a long-haul network. Summary The main objective of Resilient Packet Ring technology is to enable a true alternative to SONET transport for packet networks, providing carriers with resiliency, fast protection and restoration, and performance monitoring comparable to those of SONET networks. RPR was designed to combine SONET strengths of high availability, reliability and TDM services support, with Ethernet s low-cost, superior bandwidth utilization and high service granularity characteristics. Unlike SONET, RPR provides an Ethernet-like cost curve as well as superior bandwidth utilization, both in its Ethernet-like statistical multiplexing, and in its spatial reuse capabilities. Spatial reuse provides an extremely effi cient use of shared media traffi c in metro/access rings, since it is expected that in the near future most traffi c originating in a metro area will remain within the same metro/access ring. Unlike next-generation SONET solutions that integrate both transport and data switching in the same network element, RPR is a transport technology that fi ts into existing carriers operations model, this tremendously reduces the required operational expenses in deployment as well as the maintenance expenses associated with the manual provisioning process of today s transport networks. Unlike Ethernet transport technology, RPR provides fi ve-nines availability using SONET-grade fast protection and restoration, carrier-class fairness, the ability to transparently carry and groom TDM traffi c, and SONET-like reliability and performance monitoring capabilities. RPR is being promoted and standardized by industry leaders as well as by innovative startup companies, and is positioned to take a major role in deployment of next generation carrier-class networks. 101 Metro Drive, Suite 680 San Jose, California 95110 Phone: 408.392.9292 Fax: 408.392.9294 www.corrigent.com Copyright, Inc. All Rights Reserved Page 9 of 9