October 18, This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Transcription

1 PANRG Internet-Draft Intended status: Informational Expires: April 21, 2019 T. Enghardt TU Berlin C. Kraehenbuehl ETH Zuerich October 18, 2018 A Vocabulary of Path Properties draft-enghardt-panrg-path-properties-00 Abstract This document defines and categorizes information about Internet paths that an endpoint might have or want to have. This information is expressed as properties of paths between two endpoints. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 21, Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust s Legal Provisions Relating to IETF Documents ( in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Enghardt & Kraehenbuehl Expires April 21, 2019 [Page 1]

2 Internet-Draft Path Properties October 2018 Table of Contents 1. Introduction Domain Properties Backbone Properties Dynamic Properties Security Considerations IANA Considerations Informative References Acknowledgments Authors Addresses Introduction Because the current Internet provides an IP-based best-effort bit pipe, endpoints have little information about paths to other endpoints. A Path Aware Network exposes information about one or multiple paths through the network to endpoints, so that endpoints can use this information. Such path properties may be relatively dynamic, e.g. current Round Trip Time, close to the origin, e.g. nature of the access technology on the first hop, or far from the origin, e.g. list of ASes traversed. Usefulness over time is fundamentally different for dynamic and nondynamic properties. The merit of a momentary measurement of a dynamic path property diminishes greatly as time goes on, e.g. the merit of an RTT measurement from a few seconds ago is quite small, while a non-dynamic path property might stay relevant, e.g. a NAT can be assumed to stay on a path during the lifetime of a connection, as the removal of the NAT would break the connection. Non-dynamic properties are further separated into (local) domain properties related to the first few hops of the connection, and backbone properties related to the remaining hops. Domain properties expose a high amount of information to endpoints and strongly influence the connection behavior while there is little influence and information about backbone properties. Dynamic properties are not separated into domain and backbone properties, since most of these properties are defined for a complete path and it is difficult and seldom useful to define them on part of the path. There are exceptions such as dynamic wireless access properties, but these do not justify separation into different categories. Enghardt & Kraehenbuehl Expires April 21, 2019 [Page 2]

3 Internet-Draft Path Properties October 2018 This document addresses the first of the questions in Path-Aware Networking [I-D.irtf-panrg-questions], which is a product of the PANRG in the IRTF. 2. Domain Properties Domain path properties usually relate to the access network within the first hop or the first few hops. Endpoints can influence domain properties for example by switching from a WiFi to a cellular interface, changing their data plan to increase throughput, or moving closer to a wireless access point which increases the signal strength. A large amount of information about domain properties exists. Properties related to configuration can be queried using provisioning domains (PvDs). A PvD is a consistent set of network configuration information as defined in [RFC7556], e.g., relating to a local network interface. This may include source IP address prefixes, IP addresses of DNS servers, name of an HTTP proxy server, DNS suffixes associated with the network, or default gateway IP address. As one PvD is not restricted to one local network interface, a PvD may also apply to multiple paths. Access Technology present on the path: The lower layer technology on the first hop, for example, WiFi, Wired Ethernet, or Cellular. This can also be more detailed, e.g., further specifying the Cellular as 2G, 3G, 4G, or 5G technology, or the WiFi as a, b, g, n, or ac. These are just examples, this list is not exhaustive, and there is no common index of identifiers here. Note that access technologies further along the path may also be relevant, e.g., a cellular backbone is not only the first hop, and there may be a DSL line behind the WiFi. Monetary Cost: This is information related to billing, data caps, etc. It could be the allowed monthly data cap, the start and end of a billing period, the monetary cost per Megabyte sent or received, etc. 3. Backbone Properties Backbone path properties relate to non-dynamic path properties that are not within the endpoint s domain. They are likely to stay constant within the lifetime of a connection, since Internet "backbone" routes change infrequently. These properties usually change on the timescale of seconds, minutes, or hours, when the route changes. Enghardt & Kraehenbuehl Expires April 21, 2019 [Page 3]

4 Internet-Draft Path Properties October 2018 Even if these properties change, endpoints can neither specify which backbone nodes to use, nor verify data was sent over these nodes. An endpoint can for example choose its access provider, but cannot choose the backbone path to a given destination since the access provider will make their own policy-based routing decision. Presence of certain device on the path: Could be the presence of a certain kind of middlebox, e.g., a proxy, a firewall, a NAT. Presence of a packet forwarding node or specific Autonomous System on a path: Indicates that traffic goes through a certain node or AS, which might be relevant for deciding the level of trust this path provides. Disjointness: How disjoint a path is from another path. Path MTU: The end-to-end maximum transmission unit in one packet. Transport Protocols available: Whether a specific transport protocol can be used to establish a connection over this path. An endpoint may know this because it has cached whether it could successfully establish, e.g., a QUIC connection, or an MPTCP subflow. Protocol Features available: Whether a specific feature within a protocol is known to work over this path, e.g., ECN, or TCP Fast Open. 4. Dynamic Properties Dynamic Path Properties are expected to change on the timescale of milliseconds. They usually relate to the state of the path, such as the currently available end-to-end bandwidth. Some of these properties may depend only on the first hop or on the access network, some may depend on the entire path. Typically, Dynamic Properties can only be approximated and sampled, and might be made available in an aggregated form, such as averages or minimums. Dynamic Path Properties can be measured by the endpoint itself or somethere in the network. See [ANRW18-Metrics] for a discussion of how to measure some dynamic path properties at the endpoint. These properties may be symmetric or asymmetric. For example, an asymmetric property may be different in the upstream direction and in the downstream direction from the point of view of a particular host. Enghardt & Kraehenbuehl Expires April 21, 2019 [Page 4]

5 Internet-Draft Path Properties October 2018 Available bandwidth: Maximum number of bytes per second that can be sent or received over this path. This depends on the available bandwidth at the bottleneck, and on crosstraffic. Round Trip Time: Time from sending a packet to receiving a response from the remote endpoint. Round Trip Time variation: Disparity of Round Trip Time values either over time or among multiple concurrent connections. A high RTT variation often indicates congestion. Packet Loss: Percentage of sent packets that are not received on the other end. Congestion: Whether there is any indication of congestion on the path. Wireless Signal strength: Power level of the wireless signal being received. Lower signal strength, relative to the same noise floor, is correlated with higher bit error rates and lower modulation rates. Wireless Modulation Rate: Modulation bitrate of the wireless signal. The modulation rate determines how many bytes are transmitted within a symbol on the wireless channel. A high modulation rate leads to a higher possible bitrate, given sufficient signal strength. Wireless Channel utilization: Percentage of time during which there is a transmission on the wireless medium. A high channel utilization indicates a congested wireless network. 5. Security Considerations Some of these properties may have security implications for endpoints. For example, a corporate policy might require to have a firewall on the path. For properties provided by the network, their authenticity and correctness may need to be verified by an endpoint. 6. IANA Considerations This document has no IANA actions. Enghardt & Kraehenbuehl Expires April 21, 2019 [Page 5]

6 Internet-Draft Path Properties October Informative References [ANRW18-Metrics] Enghardt, T., Tiesel, P., and A. Feldmann, "Metrics for access network selection", Proceedings of the Applied Networking Research Workshop on - ANRW 18, DOI / , [I-D.irtf-panrg-questions] Trammell, B., "Open Questions in Path Aware Networking", draft-irtf-panrg-questions-00 (work in progress), April [RFC7556] Anipko, D., Ed., "Multiple Provisioning Domain Architecture", RFC 7556, DOI /RFC7556, June 2015, < Acknowledgments Thanks to Paul Hoffman for feedback and editorial changes. Authors Addresses Theresa Enghardt TU Berlin theresa@inet.tu-berlin.de Cyrill Kraehenbuehl ETH Zuerich cyrill.kraehenbuehl@inf.ethz.ch Enghardt & Kraehenbuehl Expires April 21, 2019 [Page 6]

7 PANRG S. Dawkins, Ed. Internet-Draft Huawei Technologies Intended status: Informational October 15, 2018 Expires: April 18, 2019 Path Aware Networking: Obstacles to Deployment (A Bestiary of Roads Not Taken) draft-irtf-panrg-what-not-to-do-00 Abstract At the first meeting of the Path Aware Networking Research Group, Oliver Bonaventure led a discussion of mostly-unsuccessful attempts to exploit Path Awareness to achieve a variety of goals, for a variety of reasons, over the past decade. At the end of that discussion, the research group agreed to catalog and analyze these ideas, in order to extract insights and lessons for path-aware networking researchers. This document contains that catalog and analysis. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 18, Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust s Legal Provisions Relating to IETF Documents ( in effect on the date of publication of this document. Please review these documents Dawkins Expires April 18, 2019 [Page 1]

8 carefully, as they describe your rights and restrictions with respect to this document. Table of Contents 1. Introduction About this Document A Note for the Editor Architectural Guidance Summary of Lessons Learned Template for Contributions Contributions Integrated Services (IntServ) Reasons for Non-deployment Lessons Learned Quick-Start TCP Reasons for Non-deployment Lessons Learned Triggers for Transport (TRIGTRAN) Reasons for Non-deployment Lessons Learned Shim Reasons for Non-deployment Lessons Learned Next Steps in Signaling (NSIS) Reasons for Non-deployment Lessons Learned Security Considerations IANA Considerations Acknowledgments Informative References Author s Address Introduction At IETF 99, the Path Aware Networking Research Group [PANRG] held its first meeting [PANRG-99], and the first presentation in that session was "A Decade of Path Awareness" [PATH-Decade]. At the end of this discussion, two things were abundantly clear. o The Internet community has accumulated considerable experience with many Path Awareness ideas over a long period of time, and o Although some Path Awareness ideas have been successfully deployed (for example, Differentiated Services, or DiffServ [RFC2475]), most of these ideas haven t seen widespread adoption. The reasons for non-adoption are many, and are worthy of study. Dawkins Expires April 18, 2019 [Page 2]

9 The meta-lessons from that experience were o Path Aware Networking is more Research than Engineering, so establishing an IRTF Research Group for Path Aware Networking is the right thing to do [RFC7418]. o Analyzing a catalog of past experience to learn the reasons for non-adoption would be a great first step for the Research Group. Allison Mankin, the IRTF Chair, officially chartered the Path Aware Networking Research Group in July, This document contains the analysis performed by that research group (see Section 2), based on that catalog (see Section 4) About this Document This document is not intended to catalog every idea about Path Aware Networking that we can find. Instead, we include enough ideas to provide background for new lessons to guide researchers in their work, in order to add those lessons to Section 2. There is no shame to having an idea included in this document. As shown in Section 2, the quality of specific proposals had little to do with whether they were deployed or not. The first contribution added to this document was for a proposal from the editor of this document Section 4.3, and it wasn t deployed. When these proposals were made, the proponents were trying to engineer something when they should have been trying to research it. Actual shame would be failing to learn from experience, and failing to share that experience with other networking researchers and engineers. We may stand on the shoulders of giants, but most of those giants Path Aware Networking ideas weren t deployed, either! Discussion of specific contributed experiences and this document in general should take place on the PANRG mailing list A Note for the Editor (Remove after taking these actions) The to-do list for upcoming revisions includes o Confirm that the Summary of Lessons Learned makes sense and is complete, in consultation with the Research Group. Dawkins Expires April 18, 2019 [Page 3]

10 o If the Research Group identifies technologies that provided lessons that aren t included in Section 2, solicit contributions for those technologies. o Edit the contributed subsections for basic consistency (since they have different contributors providing initial material) Architectural Guidance As background for understanding the Lessons Learned contained in this document, the reader is encouraged to become familiar with the Internet Architecture Board s documents on "What Makes for a Successful Protocol?" [RFC5218] and "Planning for Protocol Adoption and Subsequent Transitions" [RFC8170]. Although these two documents do not specifically target path-aware networking protocols, they are helpful resources on successful protocol adoption and deployment. Because there is an economic aspect to decisions about deployment, the IAB Workshop on Internet Technology Adoption and Transition [ITAT] report [RFC7305] also provides food for thought. 2. Summary of Lessons Learned This section summarizes the Lessons Learned from the contributed sections in Section 4. Each Lesson Learned is tagged with one or more contributions that encountered this obstacle as a significant impediment to deployment. Other contributed technologies may have also encountered this obstacle, but this obstacle may not have been the biggest impediment to deployment. o The benefit of Path Awareness has to be great enough to overcome entropy for already-deployed devices. The colloquial American English expression, "If it ain t broke, don t fix it" is a "best current practice" on today s Internet. (See Section 4.2 and Section 4.3). o Providing benefits for early adopters is key - if everyone must deploy a technology in order for the topology to provide benefits, or even to work at all, the technology is unlikely to be adopted. (See Section 4.1 and Section 4.2). o "Follow the money." If operators can t charge for a Path Aware technology in order to recover the costs of deploying it, the Dawkins Expires April 18, 2019 [Page 4]

11 benefits to the operator must be really significant. (See Section 4.3). o Impact of a Path Aware technology on operational practices can prevent deployment of promising technology. (See Section 4.4). o Per-connection state in intermediate devices is an impediment to adoption and deployment. (See Section 4.1). o Modern routers aren t designed to make heavy use of in-band signaling using mechanisms such as IPv4 and IPv6 Router Alert Options (RAO), so operators are reluctant to deploy technologies that rely on these signals. (See Section 4.5). o If endpoints can t be trusted, operators are reluctant to deploy technologies that rely on endpoints sending unauthenticated control signals to routers. (See Section 4.5). o If intermediate devices along the path can t be trusted, it s unlikely that endpoints will rely on signals from intermediate devices to drive changes to endpoint behaviors. (See Section 4.3). o The Internet is a distributed system, so the more a technology relies on information propagated from distant hosts and routers, the less likely that information is to be accurate. (See Section 4.2). o Transport protocol technologies may require information from applications, in order to work effectively, but applications may not know the information they need to provide. (See Section 4.2). 3. Template for Contributions There are many things that could be said about the Path Aware networking technologies that have been developed. For the purposes of this document, contributors are requested to provide o the name of a technology, including an abbreviation if one was used o if available, a long-term pointer to the best reference describing the technology o a short description of the problem the technology was intended to solve Dawkins Expires April 18, 2019 [Page 5]

12 o a short description of the reasons why the technology wasn t adopted o a short statement of the lessons that researchers can learn from our experience with this technology. This document is being built collaboratively. To contribute your experience, please send a Github pull request to 4. Contributions Additional contributions that provide Lessons Learned beyond those already captured in Section 2 are welcomed Integrated Services (IntServ) The suggested references for IntServ are: o RFC 1633 Integrated Services in the Internet Architecture: an Overview [RFC1633] o RFC 2211 Specification of the Controlled-Load Network Element Service [RFC2211] o RFC 2212 Specification of Guaranteed Quality of Service [RFC2212] o RFC 2215 General Characterization Parameters for Integrated Service Network Elements [RFC2215] o RFC 2205 Resource ReSerVation Protocol (RSVP) [RFC2205] In 1994, when the IntServ architecture document [RFC1633] was published, real-time traffic was first appearing on the Internet. At that time, bandwidth was a scarce commodity. Internet Service Providers built networks over DS3 (45 Mbps) infrastructure, and subrate (< 1 Mpbs) access was common. Therefore, the IETF anticipated a need for a fine-grained QoS mechanism. In the IntServ architecture, some applications require service guarantees. Therefore, those applications use the Resource Reservation Protocol (RSVP) [RFC2205] to signal bandwidth reservations across the network. Every router in the network maintains per-flow state in order to a) perform call admission control and b) deliver guaranteed service. Dawkins Expires April 18, 2019 [Page 6]

13 Applications use Flow Specification (Flow Specs) [RFC2210] to describe the traffic that they emit. RSVP reserves bandwidth for traffic on a per Flow Spec basis Reasons for Non-deployment IntServ was never widely deployed because of its cost. The following factors contributed to cost: o IntServ must be deployed on every router within the QoS domain o IntServ maintained per flow state As IntServ was being discussed, the following occurred: o It became more cost effective to solve the QoS problem by adding bandwidth. Between 1994 and 2000, Internet Service Providers upgraded their infrastructures from DS3 (45 Mbps) to OC-48 (2.4 Gbps). This meant that even if an endpoint was using IntServ in an IntServ-enabled network, its requests would never be denied, so endpoints and Internet Service Providers had little reason to enable IntServ. o DiffServ [RFC2475] offered a more cost-effective, albeit less fine-grained, solution to the QoS problem Lessons Learned. The following lessons were learned: o Any mechanism that requires a router to maintain per-flow state is not likely to succeed. o Any mechanism that requires an operator to upgrade all of its routers is not likely to succeed. IntServ was never widely deployed. However, the technology that it produced was deployed for reasons other than bandwidth management. RSVP is widely deployed as an MPLS signaling mechanism. BGP uses Flow Specs to distribute firewall filters Quick-Start TCP The suggested references for Quick-Start TCP are: o RFC 4782 Quick-Start for TCP and IP [RFC4782] Dawkins Expires April 18, 2019 [Page 7]

14 o Determining an appropriate sending rate over an underutilized network path [SAF07] o Fast Startup Internet Congestion Control for Broadband Interactive Applications [Sch11] Quick-Start [RFC4782] is an experimental TCP extension that leverages support from the routers on the path to determine an allowed sending rate, either at the start of data transfers or after idle periods. In these cases, a TCP sender cannot easily determine an appropriate sending rate, given the lack of information about the path. The default TCP congestion control therefore uses the time-consuming slow-start algorithm. With Quick-Start, connections are allowed to use higher sending rates if there is significant unused bandwidth along the path, and if the sender and all of the routers along the path approve the request. By examining Time To Live (TTL) fields, a sender can determine if all routers have approved the Quick-Start request. The protocol also includes a nonce that provides protection against cheating routers and receivers. If the Quick-Start request is explicitly approved by all routers along the path, the TCP host can send at up to the approved rate; otherwise TCP would use the default congestion control. Quick-Start requires modifications in the involved end-systems as well in routers. Due to the resulting deployment challenges, Quick-Start was only proposed in [RFC4782] for controlled environments. The Quick-Start protocol is a lightweight, coarse-grained, in-band, network-assisted fast startup mechanism. The benefits are studied by simulation in a research paper [SAF07] that complements the protocol specification. The study confirms that Quick-Start can significantly speed up mid-sized data transfers. That paper also presents router algorithms that do not require keeping per-flow state. Later studies [Sch11] comprehensively analyzes Quick-Start with a full Linux implementation and with a router fast path prototype using a network processor. In both cases, Quick-Start could be implemented with limited additional complexity Reasons for Non-deployment However, the experiments with Quick-Start in [Sch11] reveal several challenges: o Having information from the routers along the path can reduce the risk of congestion, but cannot avoid it entirely. Determining whether there is unused capacity is not trivial in actual router and host implementations. Data about available bandwidth visible at the IP layer may be imprecise, and due to the propagation delay, information can already be outdated when it reaches the Dawkins Expires April 18, 2019 [Page 8]

15 sender. There is a trade-off between the speedup of data transfers and the risk of congestion even with Quick-Start. o For scalable router fast path implementation, it is important to enable parallel processing of packets, as this is a widely used method e.g. in network processors. One challenge is synchronization of information between different packets, which should be avoided as much as possible. o Only selected applications can benefit from Quick-Start. For achieving an overall benefit, it is important that senders avoid sending unnecessary Quick-Start requests, e.g. for connections that will only send a small amount of data. This typically requires application-internal knowledge. It is a mostly unsolved question how a sender can indeed determine the data rate that Quick-Start shall request for. After completion of the Quick-Start specification, there have been large-scale experiments with an initial window of up to 10 MSS [RFC6928]. This alternative "IW10" approach can also ramp up data transfers faster than the standard TCP congestion control, but it only requires sender-side TCP modifications. As a result, this approach can be easier and incrementally deployed in the Internet. While theoretically Quick-Start can outperform "IW10", the absolute improvement of data transfer times is rather small in many cases. After publication of [RFC6928], most modern TCP stacks have increased their default initial window. There is no known deployment of Quick- Start TCP Lessons Learned There are some lessons learned from Quick-Start. Despite being a very light-weight protocol, Quick-Start suffers from poor incremental deployment properties, both regarding the required modifications in network infrastructure as well as its interactions with applications. Except for corner cases, congestion control can be quite efficiently performed end-to-end in the Internet, and in modern TCP stacks there is not much room for significant improvement by additional network support Triggers for Transport (TRIGTRAN) The suggested references for TRIGTRAN are: o TRIGTRAN BOF at IETF 55 [TRIGTRAN-55] o TRIGTRAN BOF at IETF 56 [TRIGTRAN-56] Dawkins Expires April 18, 2019 [Page 9]

16 TCP [RFC0793] has a well-known weakness - the end-to-end flow control mechanism has only a single signal, the loss of a segment, and semimodern TCPs (since the late 1980s) have interpreted the loss of a segment as evidence that the path between two endpoints has become congested enough to exhaust buffers on intermediate hops, so that the TCP sender should "back off" - reduce its sending rate until it knows that its segments are now being delivered without loss [RFC2581]. More modern TCPs have added a growing array of strategies about how to establish the sending rate [RFC5681], but when a path is no longer operational, TCPs can wait many seconds before retrying a segment, even if the path becomes operational while the sender is waiting for its next retry. The thinking in Triggers for Transport was that if a path completely stopped working because its first-hop link was "down", that somehow TCP could be signaled when the first-hop link returned to service, and the sending TCP could retry immediately, without waiting for a full Retransmission Time Out (RTO) period Reasons for Non-deployment Two TRIGTRAN BOFs were held, at IETF 55 [TRIGTRAN-55] and IETF 56 [TRIGTRAN-56], but this work was not chartered, and there was no interest in deploying TRIGTRAN unless it was chartered and standardized in the IETF Lessons Learned. The reasons why this work was not chartered provide several useful lessons for researchers. o TRIGTRAN triggers are only provided when the first-hop link is "down", so TRIGTRAN triggers couldn t replace normal TCP retransmission behavior if the path failed because some link further along the network path was "down". So TRIGTRAN triggers added complexity to an already complex TCP state machine, and didn t allow any existing complexity to be removed. o The state of the art in the early 2000s was that TRIGTRAN triggers were assumed to be unauthenticated, so they couldn t be trusted to tell a sender to "speed up", only to "slow down". This reduced the potential benefit to implementers. o intermediate forwarding devices required modification to provide TRIGTRAN triggers, but operators couldn t charge for TRIGTRAN triggers, so there was no way to recover the cost of modifying, testing, and deploying updated intermediate devices. Dawkins Expires April 18, 2019 [Page 10]

17 4.4. Shim6 The suggested references for Shim6 are: o RFC5533 Shim6: Level 3 Multihoming Shim Protocol for IPv6 [RFC5533] The IPv6 routing architecture [RFC1887] assumed that most sites on the Internet would be identified by Provider Assigned IPv6 prefixes, so that Default-Free Zone routers only contained routes to other providers, resulting in a very small routing table. For a single-homed site, this could work well. A multi-homed site with only one upstream provider could also work well, although BGP multihoming from a single upstream provider was often a premium service (costing more than twice as much as two single-homed sites), and if the single upstream provider went out of service, all of the multi-homed paths could fail simultaneously. IPv4 sites often multihomed by obtaining Provider Independent prefixes, and advertising these prefixes through multiple upstream providers. With the assumption that any multihomed IPv4 site would also multihome in IPv6, it seemed likely that IPv6 routing would be subject to the same pressures to announce Provider Independent prefixes, resulting in a global IPv6 routing table that exhibited the same problems as the global IPv4 routing table. During the early 2000s, work began on a protocol that would provide the same benefits for multihomed IPv6 sites without requiring sites to advertise Provider Independent prefixes into the global routing table. This protocol, called Shim6, allowed two endpoints to exchange multiple addresses ("Locators") that all mapped to the same endpoint ("Identity"). After an endpoint learned multiple Locators for the other endpoint, it could send to any of those Locators with the expectation that those packets would all be delivered to the endpoint with the same Identity. Shim6 was an example of an "Identity/Locator Split" protocol. Shim6, as defined in [RFC5533] and related RFCs, provided a workable solution for IPv6 multihoming using Provider Assigned prefixes, including capability discovery and negotiation, and allowing end-toend application communication to continue even in the face of path failure, because applications don t see Locator failures, and continue to communicate with the same Identity using a different Locator. Dawkins Expires April 18, 2019 [Page 11]

18 Reasons for Non-deployment Note that the problem being addressed was "site multihoming", but Shim6 was providing "host multihoming". That meant that the decision about what path would be used was under host control, not under router control. Although more work could have been done to provide a better technical solution, the biggest impediments to Shim6 deployment were operational and business considerations. These impediments were discussed at multiple network operator group meetings, including [Shim6-35] at [NANOG-35]. The technology issues centered around scaling concerns that Shim6 relied on the host to track all the TCP connections and the file descriptions with associated HTTP state, while also tracking Identity/Locator mappings in the kernel, and tracking failures to recognize that a backup path has failed. The operator issues centered around concerns that operators were performing traffic engineering, but would have no visibility or control over hosts when they chose to begin using another path, and relying on hosts to engineer traffic exposed their networks to oscillation based on feedback loops, as hosts move from path to path. At a minimum, traffic engineering policies must be pushed down to individual hosts. In addition, the usual concerns about firewalls that expected to find a transport-level protocol header in the IP payload, and won t be able to perform firewalling functions because its processing logic would have to look past the Identity header. The business issues centered removing or reducing the ability to sell BGP multihoming service, which is often more expensive than singlehomed connectivity Lessons Learned It is extremely important to take operational concerns into account when a path-aware protocol is making decisions about path selection that may conflict with existing operational practices and business considerations. We also note that some path-aware networking ideas recycle. Stream Control Transmission Protocol (SCTP) has provided support for multihoming since 2000 [RFC2960], but was designed to transport PSTN signaling messages over IP networks. SCTP was capable of broader applications, but because multi-homed hosts in the 1990s were uncommon, and deployment of new transport protocols such as SCTP required either operating system kernel support or access to raw IP Dawkins Expires April 18, 2019 [Page 12]

19 packets until a UDP encapsulation for SCTP [RFC6951] was produced in 2013, SCTP multihoming did not stir up the same operator concerns that Shim6 encountered. Although Shim6 did not achieve significant deployment, the IETF chartered a working group to specify "Multipath TCP" [MP-TCP] in 2009, and Multipath TCP allows general-purpose TCP applications to control path selection, with many of the same advantages and disadvantages of Shim Next Steps in Signaling (NSIS) The suggested references for NSIS are: o the concluded working group charter [NSIS-CHARTER-2001] o RFC 5971 GIST: General Internet Signalling Transport [RFC5971] o RFC 5973 NAT/Firewall NSIS Signaling Layer Protocol (NSLP) [RFC5973] o RFC 5974 NSIS Signaling Layer Protocol (NSLP) for Quality-of- Service Signaling [RFC5974] o RFC 5981 "Authorization for NSIS Signaling Layer Protocols [RFC5981] The Next Steps in Signaling (NSIS) Working Group worked on signaling technologies for network layer resources (e.g., QoS resource reservations, Firewall and NAT traversal). When RSVP [RFC2205] was used in deployments, a number of questions came up about its perceived limitations and potential missing features. The issues noted in the NSIS Working Group charter [NSIS-CHARTER-2001] include interworking between domains with different QoS architectures, mobility and roaming for IP interfaces, and complexity. Later, the lack of security in RSVP was also recognized ([RFC4094]). The NSIS Working Group was chartered to tackle those issues and initially focused on QoS signaling as its primary use case. However, over time a new approach evolved that introduced a modular architecture using application-specific signaling protocols (the NSIS Signaling Layer Protocol (NSLP)) on top of a generic signaling transport protocol (the NSIS Transport Layer Protocol (NTLP)). The NTLP is defined in [RFC5971]. Two NSLPs are defined: the NSIS Signaling Layer Protocol (NSLP) for Quality-of-Service Signaling [RFC5974] as well as the NAT/Firewall NSIS Signaling Layer Protocol (NSLP) [RFC5973]. Dawkins Expires April 18, 2019 [Page 13]

20 Reasons for Non-deployment The obstacles for deployment can be grouped into implementationrelated aspects and operational aspects. o Implementation-related aspects: Although NSIS provides benefits with respect to flexibility, mobility, and security compared to other network signaling technologies, hardware vendors were reluctant to deploy this solution, because it would require additional implementation effort and would result in additional complexity for router implementations. The NTLP mainly operates as path-coupled signaling protocol, i.e, its messages are processed at the intermediate node s control plane that are also forwarding the data flows. This requires a mechanism to intercept signaling packets while they are forwarded in the same manner (especially along the same path) as data packets. One reason for the non-deployment of NSIS is the usage of the IPv4 and IPv6 Router Alert Option (RAO) to allow for an efficient interception of those path-coupled signaling messages: This option requires router implementations to correctly understand and implement the handling of RAOs, e.g., to only process packet with RAOs of interest and to leave packets with irrelevant RAOs in the fast forwarding processing path (a comprehensive discussion of these issues can be found in [RFC6398]). The latter was an issue with some router implementations at the time of standardization. Another reason is that path-coupled signaling protocols that interact with routers and request manipulation of state at these routers (or any other network element in general) are under scrutiny: a packet (or sequence of packets) out of the mainly untrusted data path is requesting creation and manipulation of network state. This is seen as potentially dangerous (e.g., opens up a Denial of Service (DoS) threat to a router s control plane) and difficult for an operator to control. End-to-end signaling approaches were considered problematic (see also section 3 of [RFC6398]). There are recommendations on how to secure NSIS nodes and deployments (e.g., [RFC5981]). o Operational Aspects: End-to-end signaling technologies not only require trust between customers and their provider, but also among different providers. Especially, QoS signaling technologies would require some kind of dynamic service level agreement support that would imply (potentially quite complex) bilateral negotiations between different Internet service providers. This complexity was currently not considered to be justified and increasing the bandwidth capacity (and thus avoiding Dawkins Expires April 18, 2019 [Page 14]

21 bottlenecks) was cheaper than actively managing network resource bottlenecks by using path-coupled QoS signaling technologies. Furthermore, an end-to-end path typically involves several provider domains and these providers need to closely cooperate in cases of failures Lessons Learned One goal of NSIS was to decrease the complexity of the signaling protocol, but a path-coupled signaling protocol comes with the intrinsic complexity of IP-based networks, beyond the complexity of the signaling protocol itself. Sources of intrinsic complexity include o the presence of asymmetric routes between endpoints and routers o the lack of security and trust at large in the Internet infrastructure o the presence of different trust boundaries, o the effects of best-effort networks (e.g., packet loss) o divergence from the fate sharing principle (e.g., state within the network). Any path-coupled signaling protocol has to deal with these realities. Operators view the use of IPv4 and IPv6 Router Alert Option (RAO) to signal routers along the path from end systems with suspicion, because these end systems are usually not authenticated and heavy use of RAOs can easily increase the CPU load on routers that are designed to process most packets using a hardware "fast path". 5. Security Considerations This document describes ideas that were not adopted and widely deployed on the Internet, so it doesn t affect the security of the Internet. If this document meets its goals, we may develop new ideas for Path Aware Networking that would affect the security of the Internet, but security considerations for those ideas will be described in the corresponding RFCs that propose them. Dawkins Expires April 18, 2019 [Page 15]

22 6. IANA Considerations This document makes no requests of IANA. 7. Acknowledgments Initial material for Section 4.1 on IntServ was provided by Ron Bonica. Initial material for Section 4.2 on Quick-Start TCP was provided by Michael Scharf. Initial material for Section 4.3 on Triggers for Transport (TRIGTRAN) was provided by Spencer Dawkins. Section 4.4 on Shim6 builds on initial material describing obstacles provided by Erik Nordmark, with background added by Spencer Dawkins. Initial material for Section 4.5 on Next Steps In Signaling (NSIS) was provided by Roland Bless and Martin Stiemerling. Our thanks to Roland Bless, Ruediger Geib, and Joe Touch, who provided review comments on previous versions. 8. Informative References [ITAT] "IAB Workshop on Internet Technology Adoption and Transition (ITAT)", December 2013, < [MP-TCP] "Multipath TCP Working Group Home Page", n.d., < [NANOG-35] "North American Network Operators Group NANOG-35 Agenda", October 2005, < [NSIS-CHARTER-2001] "Next Steps In Signaling Working Group Charter", March 2011, < [PANRG] "Path Aware Networking Research Group (Home Page)", n.d., < Dawkins Expires April 18, 2019 [Page 16]

23 [PANRG-99] "Path Aware Networking Research Group - IETF-99", July 2017, < [PATH-Decade] Bonaventure, O., "A Decade of Path Awareness", July 2017, < slides-99-panrg-a-decade-of-path-awareness/>. [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, DOI /RFC0793, September 1981, < [RFC1633] Braden, R., Clark, D., and S. Shenker, "Integrated Services in the Internet Architecture: an Overview", RFC 1633, DOI /RFC1633, June 1994, < [RFC1887] Rekhter, Y., Ed. and T. Li, Ed., "An Architecture for IPv6 Unicast Address Allocation", RFC 1887, DOI /RFC1887, December 1995, < [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification", RFC 2205, DOI /RFC2205, September 1997, < [RFC2210] Wroclawski, J., "The Use of RSVP with IETF Integrated Services", RFC 2210, DOI /RFC2210, September 1997, < [RFC2211] Wroclawski, J., "Specification of the Controlled-Load Network Element Service", RFC 2211, DOI /RFC2211, September 1997, < [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification of Guaranteed Quality of Service", RFC 2212, DOI /RFC2212, September 1997, < [RFC2215] Shenker, S. and J. Wroclawski, "General Characterization Parameters for Integrated Service Network Elements", RFC 2215, DOI /RFC2215, September 1997, < Dawkins Expires April 18, 2019 [Page 17]

24 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., and W. Weiss, "An Architecture for Differentiated Services", RFC 2475, DOI /RFC2475, December 1998, < [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion Control", RFC 2581, DOI /RFC2581, April 1999, < [RFC2960] Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., Zhang, L., and V. Paxson, "Stream Control Transmission Protocol", RFC 2960, DOI /RFC2960, October 2000, < [RFC4094] Manner, J. and X. Fu, "Analysis of Existing Quality-of- Service Signaling Protocols", RFC 4094, DOI /RFC4094, May 2005, < [RFC4782] Floyd, S., Allman, M., Jain, A., and P. Sarolahti, "Quick- Start for TCP and IP", RFC 4782, DOI /RFC4782, January 2007, < [RFC5218] Thaler, D. and B. Aboba, "What Makes for a Successful Protocol?", RFC 5218, DOI /RFC5218, July 2008, < [RFC5533] Nordmark, E. and M. Bagnulo, "Shim6: Level 3 Multihoming Shim Protocol for IPv6", RFC 5533, DOI /RFC5533, June 2009, < [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion Control", RFC 5681, DOI /RFC5681, September 2009, < [RFC5971] Schulzrinne, H. and R. Hancock, "GIST: General Internet Signalling Transport", RFC 5971, DOI /RFC5971, October 2010, < [RFC5973] Stiemerling, M., Tschofenig, H., Aoun, C., and E. Davies, "NAT/Firewall NSIS Signaling Layer Protocol (NSLP)", RFC 5973, DOI /RFC5973, October 2010, < Dawkins Expires April 18, 2019 [Page 18]

25 [RFC5974] Manner, J., Karagiannis, G., and A. McDonald, "NSIS Signaling Layer Protocol (NSLP) for Quality-of-Service Signaling", RFC 5974, DOI /RFC5974, October 2010, < [RFC5981] Manner, J., Stiemerling, M., Tschofenig, H., and R. Bless, Ed., "Authorization for NSIS Signaling Layer Protocols", RFC 5981, DOI /RFC5981, February 2011, < [RFC6398] Le Faucheur, F., Ed., "IP Router Alert Considerations and Usage", BCP 168, RFC 6398, DOI /RFC6398, October 2011, < [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, "Increasing TCP s Initial Window", RFC 6928, DOI /RFC6928, April 2013, < [RFC6951] Tuexen, M. and R. Stewart, "UDP Encapsulation of Stream Control Transmission Protocol (SCTP) Packets for End-Host to End-Host Communication", RFC 6951, DOI /RFC6951, May 2013, < [RFC7305] Lear, E., Ed., "Report from the IAB Workshop on Internet Technology Adoption and Transition (ITAT)", RFC 7305, DOI /RFC7305, July 2014, < [RFC7418] Dawkins, S., Ed., "An IRTF Primer for IETF Participants", RFC 7418, DOI /RFC7418, December 2014, < [RFC8170] Thaler, D., Ed., "Planning for Protocol Adoption and Subsequent Transitions", RFC 8170, DOI /RFC8170, May 2017, < [SAF07] Sarolahti, P., Allman, M., and S. Floyd, "Determining an appropriate sending rate over an underutilized network path", Computer Networking Volume 51, Number 7, May [Sch11] Scharf, M., "Fast Startup Internet Congestion Control for Broadband Interactive Applications", Ph.D. Thesis, University of Stuttgart, April Dawkins Expires April 18, 2019 [Page 19]

26 [Shim6-35] Meyer, D., Huston, G., Schiller, J., and V. Gill, "IAB IPv6 Multihoming Panel at NANOG 35", NANOG North American Network Operator Group, October 2005, < [TRIGTRAN-55] "Triggers for Transport BOF at IETF 55", July 2003, < [TRIGTRAN-56] "Triggers for Transport BOF at IETF 56", November 2003, < Author s Address Spencer Dawkins (editor) Huawei Technologies spencerdawkins.ietf@gmail.com Dawkins Expires April 18, 2019 [Page 20]

27 TSVWG Y. Li INTERNET-DRAFT X. Zhou Intended Status: Informational Huawei Expires: April 12, 2019 October 9, 2018 Overlayed Path Segment Forwarding (OPSF) Problem Statement draft-li-tsvwg-overlayed-path-segment-fwding-ps-00 Abstract Various overlays are used in networks including WAN, enterprise campus and others. End to end path are divided into multiple segments some of which are overlay encapsulated to achieve better path selection, lower latency and so on. Traditional end-to-end transport layer is not very responding to microburst and non-congestive packet loss caused by the different characteristics of the path segments. With the potential transport enhancement for the existing or purposely created overlayed path segment, end to end throughput can be improved. This document illustrates the problems in some use cases and tries to inspire more about whether and how to solve them by introducing a reliable, efficient and non-intrusive transport forwarding over the overlayed path segment(s). Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at The list of Internet-Draft Shadow Directories can be accessed at Li, et al [Page 1]