Internet Peering Agreements Evolve

Transcription

1 Internet Peering Agreements Evolve Research Brief Abstract: Gartner Dataquest examines the dynamics of traffic exchange between Internet backbone providers, and the revenue implications for network service providers. By Lydia Leong Recommendations ISPs that do not have formalized peering agreements should seek to formalize those relationships. ISPs and carriers should optimize their peering relationships to maximize revenue from settlements. ISPs that purchase transit or pay-equivalent settlement fees should seek direct traffic exchange relationships from the content providers that are most popular with their subscribers. Publication Date: September 26, 2002

2 2 Internet Peering Agreements Evolve Introduction The Internet is a network of networks a conglomeration of individual networks run by individual companies and interconnected via a standardized set of protocols. Consequently, no one company or country "owns" the Internet, and the Internet has very limited governance. Thus, the interconnection of individual networks is a private matter, not a public one; the agreements that govern traffic exchange between networks, which are known as "peering agreements," can be as formal or informal as the network owners wish, and on whatever economic terms are agreeable to both parties. Since 1995, the following crucial changes to peering have occurred: Most peering is private, rather than public. Peering is governed by legal contracts, not informal "handshake" agreements. Peering is settlement-based, with an agreed-upon price for imbalances in traffic exchange, rather than being a free courtesy between network providers. Content providers are peering directly with Internet service providers (ISPs), rather than purchasing transit services. These peering trends have implications for carriers, ISPs and operators of Internet data centers, as well as content providers. Furthermore, they have economic implications for developing countries whose Internet infrastructure is currently limited. A Technical Overview of Peering The Internet is a collection of networks known as "autonomous systems." An autonomous system is a virtual construct; it is not defined by physical fiber, but rather, by routing policy. An autonomous system is a network or group of networks owned and managed by a single entity, such as a carrier, an ISP, an enterprise or a university. For one autonomous system to directly move packets from its network to the network of another autonomous system, the two autonomous systems must peer with each other. Autonomous systems can also reach each other via other autonomous systems. For example, if network A is peered with network B, and network B is peered with network C, network A can route traffic destined for network C to network B, and network B will pass it on to network C. Thus, the Internet is a mesh of interconnections, not a single virtual cloud. Every provider does not peer with every other provider in a given location. For instance, if enterprise A is a customer of Sprint in Detroit, and enterprise B is a customer of Verio in Detroit, but Sprint and Verio do not peer in Detroit, traffic from enterprise A to enterprise B must travel until it reaches an exchange point. If the closest peering point between Sprint and

3 3 Verio is a facility in Chicago, traffic from enterprise A might travel from Detroit to Chicago on Sprint's network, be passed from Sprint's router to Verio's router within their Chicago exchange point, and then travel on Verio's network from Chicago to Detroit, before finally arriving at enterprise B. To peer, an autonomous system must have a router that is capable of directly communicating with the router of another anonymous system; these routers are known as "border routers." The least-expensive method is for each network to come to a carrier hotel or colocation facility, place routers there, and connect the routers via a form of passive media. Alternatively, an Internet exchange (IX) may provide an asynchronous transfer mode (ATM) cloud; an autonomous system buys a circuit into the cloud and sets up virtual circuits with each other autonomous system that it wishes to peer with. It is not technically possible for ISPs to exchange traffic with each other at an unlimited number of exchange points. Such a scheme would dramatically increase the size and complexity of the global routing table, rendering it impossible for a router to determine the appropriate path for traffic in real time. A Historical Overview of Peering In 1994, most commercial ISPs were members of the Commercial Internet Exchange (CIX). CIX operated a peering point, and the ISPs who were members of CIX interconnected at CIX. In 1995, the U.S. government privatized NSFnet, the collection of government networks, educational institutions, research labs and similar organizations that had formed much of the early Internet. This was done in recognition of the fact that multiple companies had built commercial Internet backbones that stretched the breadth of the United States. As part of this privatization, four Network Access Points (NAPs) were designated for the exchange of Internet traffic between the new commercial backbones. The private Internet exchange (IX) business grew rapidly, but was centered on less than a dozen exchanges, including the NAPs; the largest were MFS's MAE-East and MAE-West. All ISP interconnections were engineered within an IX. Peering agreements were informal, often negotiated via between network engineers at the ISPs; there were few written contracts, and ISPs did not pay each other for the privilege of peering. For a peering relationship to be reasonably equitable, traffic across the peer needed to be balanced; in other words, ISP A needed to send ISP B approximately the same amount of traffic that ISP B sent ISP A. The larger existing backbone ISPs tried to persuade smaller ISPs to buy transit from them, rather than peering directly. This meant that the smaller ISP would buy a circuit from the larger ISP; although the technical

4 4 Internet Peering Agreements Evolve exchange of route information remained the same, the smaller ISP became a customer of the larger ISP and paid for the traffic across the link. The transit vs. peering arrangement, however, was not merely one of economics. At the time, ISPs that did not purchase transit were known as "Tier 1 ISPs." This was a marketing advantage, though in practice, there werevastdifferencesinthesizeandscaleofthevarioustier1isps. By 1996, ISPs had begun to move toward more formal peering policies, setting and publishing criteria for which ISPs they would peer with, and in what manner. ISPs frequently made exceptions to these policies, which were often based on requiring peering at a minimum of a certain number of major exchange points, typically five. National ISPs would also peer with those that had smaller networks, such as regional ISPs, for traffic engineering reasons. It became important for a backbone to be "balanced" to send and receive approximately equal traffic. For example, to obtain peering, a Web hoster that had vastly more outbound traffic than inbound traffic, needed to find a way to balance that traffic in other words, the Web hoster needed to find customers that would retrieve content, rather than serve content. This usually resulted in steeply discounted products that contributed to traffic balance; Exodus, for instance, sold T 1 Internet access to businesses, and Digex sold high-capacity links to consumer ISPs. Traffic on the Internet grew exponentially, and with it, so did the congestion at the major exchange points. Many ISPs with their own backbones could offer very fast performance to their customers, as long as the traffic stayed entirely within that ISP's network. Exchanging traffic with other providers, however, often induced latency and packet loss because of congestion, and the limited number of exchange points often forced traffic to traverse greater geographical distances, adding additional latency. As a result, during 1996, two major ISPs InterNAP and Savvis were founded based on an "overlay network" concept. Rather than building out their own extensive backbones, these ISPs chose to build minimal networks and then purchase extensive amounts of traffic from the largest backbone providers. This effectively permitted them to engineer a routing methodology that would quickly get their customers' traffic to the backbones for which it was destined. At the time, the size and speed of an ISP's backbone was still considered a major competitive factor; there were large, measurable differences in backbone performance. Most ISPs, however, followed a routing policy of first-exit routing, sometimes known as "hot potato routing." In first-exit routing, traffic is pushed off the originating ISP's network, onto the destination ISP's network, as quickly as possible, at the nearest peer. If ISP A's customer sent traffic destined for a customer of ISP B, ISP A would route the traffic over its own network to the closest exchange point where it peered with ISP B, and then dump the traffic onto ISP B's network, leaving it to ISP B to figure out the best way to get traffic to ISP B's customer. This type of routing, however, means that the originating ISP quickly loses control of the quality of the transaction, and its customer's experience does not truly reflect the quality of its backbone.

5 5 By 1997, ISPs had begun to utilize some best-exit routing within their networks, attempting to keep traffic that originated from their network within their network for as long as possible, before passing it off to the destination ISP. This allowed ISPs to create more consistent customer experiences, although it had effect of increasing the amount of traffic the ISP would have to carry on its own backbone. Some ISPs actively marketed their use of best-exit routing. Nonetheless, overlay networks and routing policy changes were insufficient to deal with the core problem too much traffic was being exchangedatthemajorixs.moreover,aproblemwithasingleexchange could have an enormous impact on Internet traffic, as was demonstrated in the summer of 1997, when a construction accident resulted in power being lost to half of the MAE-West exchange point. Finally, the IXs were not necessarily the most ideal geographic place for every ISP to exchange traffic, given that even many national ISPs still had a concentration of traffic in particular regions. The backbone providers began what would become a multiyear transition to private peering, often formalized with a legal, binding contract. Rather than exchange traffic within one of the major IXs, two ISPs would agree to peer in private facilities at some mutually satisfactory location, usually a carrier hotel. These ISPs would not abandon their public peers, but over time, would shift their routing policies to prefer these private peers over the public ones. This was often less expensive than peering in the largest IXs. At the MAEs, for instance, ISPs paid prices equivalent to local loop for the circuits used to transfer traffic within the facilities. In a carrier hotel, or at a carrier-neutral exchange, the ISPs only paid for colocation space. By 2001, however, the trend toward one-off private peers had partially reversed itself. The number of major backbone operators had declined since 1997, but there were still far too many to allow a given provider to engineer multiple unique peering locations with each of the other major providers. The growth of carrier-neutral exchanges, such as Equinix and PAIX, made it attractive for multiple backbone providers to choose a limited number of these locations to peer within. Each provider must still negotiate each new peer, which may carry separate terms and conditions. Providers usually do not peer with every other provider in the facility because this represents separate agreements with each of those providers and may be undesirable from a traffic engineering standpoint. Nonetheless, in 2002, there is still a significant amount of public peering, primarily between small and midsize ISPs that do not charge each other for the exchange of traffic. Major backbone providers also remain at the large IXs, though the circumstances under which they peer have changed. Gartner Dataquest Perspective The rise of private peering agreements has drastically altered the business arrangements between ISPs. Early ISPs believed that peering should be extended as a free courtesy between providers, on the basis that the

6 6 Internet Peering Agreements Evolve arrangements were reasonably equitable and of benefit to the Internet as a whole. The market has, however, evolved into providers with different inbound and outbound traffic characteristics, as follows: Narrowband consumer ISPs will primarily obtain inbound traffic. The subscribers are consuming content; they are "eyeballs." Web hosters will primarily generate outbound traffic. They are serving content. Broadband consumer ISPs will obtain massive amounts of inbound traffic because their subscribers are able to consume large amounts of content in very short time frames. These ISPs also generate some outbound traffic because of subscribers that run peer-to-peer servers at home. Business-focused ISPs will have relatively balanced traffic patterns. The mix of traffic will shift throughout the day, but overall, while business users are eyeballs, the businesses themselves also serve content, and corporate links are heavily utilized for , enterprise applications and the like. Backbone providers will have a mix of traffic determined by the relative size of their various businesses: enterprise Internet access, Web hosting, consumer narrowband and broadband Internet access on a retail and wholesale basis. The ratio of eyeballs receiving content, to bytes of content served, is now the primary driver behind modern peering agreements. Settlement-Based Peering The distinction between transit and peering has blurred. Transit no longer implies a second-tier status, nor does peering imply that the transaction is free to both parties. At present, most service providers exchange traffic with one another via direct peers, rather than utilizing transit circuits. The formal peering agreements, however, specify settlement charges, based on the traffic passed from one network to the other. If the traffic is balanced in both directions, then neither provider pays the other. If the traffic balance shifts, however, at the end of the month, the provider whose traffic has exceeded the limits will pay the other provider. This is not a customer relationship; this is an exchange of services. Gartner Dataquest believes that this type of settlement-based peering will become the norm for such transactions. In order for a peering agreement to be settlement free called "zerosettlement peering" the two ISPs involved typically must reach a percentage of each other's geographic territory, as well as meet minimums for network scope, typically measured in a combination of backbone capacity, number and circuit size of interconnections, and the number of routes announced. Traffic exchange in settlement-free peering agreements is normally approximately equal; for instance, WorldCom specifies a ratio that is not to exceed 1.5 to 1.

7 Settlement-based peering has made traffic engineering into an art. A provider's network engineers must not only be capable of operating an efficient and reliable network, but they must also be capable of creating traffic patterns that maximize their company's revenue from settlements and minimizes settlement expenses. Traffic is dynamic. The traffic across a peer can change drastically from one day to another if a provider obtains or loses a major customer whose traffic is traversing that peer. Changes can also accumulate over longer periods of time, such as when a provider turns up a new point of presence or opens a sales office in a new city. Changes occur seasonally, and there is also gradual change caused by alterations in the provider's customer base. A provider's network engineering staff must be able to react to these changes as well as be alert for opportunities to shift traffic in ways that will be economically advantageous from a settlement point of view. Note that the traffic engineering involved can sometimes have a direct impact on an ISP's relationship with an enterprise. Settlement agreements can make it advantageous for an ISP to peer directly with an enterprise, or to sell an enterprise connectivity at a reduced price. For instance, suppose that enterprise A and enterprise B are both located in the same city, and are in related businesses, resulting in a significant percentage of network traffic between the two companies. Enterprise A, however, is a customer of ISP C, while enterprise B is a customer of ISP D. Enterprise A is sending enterprise B so much traffic that ISP C is paying settlement charges to ISP D because of traffic crossing their peer in that city. It thus makes sense for ISP C to directly peer with enterprise B if physical facilities make this convenient, or to sell a circuit to Enterprise B at a sharply discounted price. That would allow traffic from A to reach enterprise B, without crossing the peer that ISP C and D have with each other. If ISP C manages to accomplish this, the traffic balance on the peer shifts, and, if the alteration is significant enough, ISP D might end up paying ISP C settlement charges. Content Provider Peering It is mutually advantageous for providers with opposing traffic patterns to peer directly with one another. For instance, the customers of a Web hoster will receive better rankings on measurement of Web performance such as those generated by Keynote, if that Web hoster has extensive peering arrangements with a wide variety of ISPs with eyeballs. Those eyeballs, in turn, will receive content more quickly, and thus have a better user experience. Few consumer ISPs own extensive networks. They are thus forced to purchase transit from a backbone provider, or pay high fees for settlement-based peering. Furthermore, content providers are paying monthly fees for network services from Web hosters. The natural evolution of this is for consumer ISPs to begin to peer directly with content providers, within a carrier-neutral exchange or Internet data center. 7

8 8 Internet Peering Agreements Evolve For instance, the vast majority of Earthlink's traffic is inbound content requested by its consumer eyeballs, while virtually all of Yahoo's traffic is outbound content from its portal services. Under normal circumstances, Earthlink purchases transit from a backbone provider, and Yahoo purchases network connectivity through its Web hoster or a carrier within a neutral exchange. If, however, Earthlink comes directly into a facility where Yahoo has servers, Earthlink can directly peer with Yahoo within that facility. There is still a network cost involved, since Earthlink is paying for colocation and network connectivity into the facility, but this saves Yahoo the need to purchase any connectivity. The two providers can split the cost of Earthlink's circuit into the facility and thus lower their costs. Gartner Dataquest believes that these arrangements will become increasingly commonplace, as they are financially advantageous to both parties. This is good news for neutral exchanges, which derive their revenue primarily from colocation fees. It is less positive for Web hosters, which derive a significant portion of their revenue from connectivity fees. It is damaging to backbone providers that would otherwise receive fees for two circuits of comparable size, rather than one essentially cutting their revenue by half. The Bottom Line Gartner Dataquest believes that settlement-based peering will be the norm for Internet traffic exchange through The mechanics of settlement will evolve, however, as providers determine how best to accurately reflect the economics of traffic exchange in settlement payments. Gartner Dataquest also believes that the revenue of carrier-neutral Internet exchanges will continue to grow, representing the bulk of revenue in unmanaged colocation in data centers by This growth will be driven not just by ISPs and carriers within those facilities, but by content providersandenterpriseswithhightrafficvolumesthatcanobtainmore favorable economic terms as well as superior network performance through purchase of bandwidth directly within those exchanges. Key Issue How will opportunities in the public network service market be affected by competition, technology and evolving user requirements?

9 9

10 10 Internet Peering Agreements Evolve This document has been published to the following Marketplace codes: TELC-WW-DP-0226 For More Information... In North America and Latin America: In Europe, the Middle East and Africa: In Asia/Pacific: In Japan: Worldwide via gartner.com: Entire contents 2002 Gartner, Inc. All rights reserved. Reproduction of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice