Segment based Inter-networking to Accommodate Diversity at the Edge

Transcription

1 Segment based Inter-networking to Accommodate Diversity at the Edge Fahad R. Dogar and Peter Steenkiste Carnegie Mellon University Abstract In this paper, we introduce Tapa, a network architecture that accommodates diversity at the network edge: different access networks, heterogeneous edge devices, and rich applications and network services. The core idea underlying Tapa is to use network segments, rather than IP links, as the basis on top of which end-to-end transport semantics are implemented. A segment spans a part of the end-to-end path that can be considered homogeneous (e.g., wired Internet or an access network) network specific functions and optimizations, like congestion control, are implemented within a segment, allowing use of customized solutions. On top of segments is the transfer layer, which manages end-toend data transfer over multiple segments, supporting better source and path selection mechanisms than IP. Finally, on top of the transfer layer is an end-to-end transport layer that mainly deals with implementing a rich set of semantics to accommodate diverse applications and network services. We implemented Tapa and show how it supports diverse applications and optimizations. 1. INTRODUCTION Today s Internet is much different from the original Internet of the 1970s. Many of these changes have occurred at the network edge, in the form of diverse Internet access technologies (e.g., blue-tooth, ultra-wide-band), edge devices (e.g., cell-phone, PDAs, sensors), applications (e.g., content sharing, gaming, sensing apps), services supported by the network (e.g., caching, mobile users), and the nature of network deployments (e.g., unmanaged residential wireless networks [3]). Supporting this diversity often requires help from the network, which is hard given the intelligent endpoints, dumb network principle followed in the Internet architecture. As a result, ad-hoc solutions, such as transparent proxies or CDNs are used, which are complex to manage and violate one or more architectural principles of the Internet [7, 11, 21, 15]. Since diversity at the edges is only going to increase in future, it is critical that we develop architectural mechanisms rather than point solutions to accommodate it in the Internet architecture. To this end, we introduce Tapa, a network architecture that addresses this problem by raising the level of abstraction on which end-to-end transport is implemented. This new abstraction, named segment, corresponds to a portion of an end-to-end path that is homogeneous. Some possible segments include: the wired Internet, a private network owned by an enterprise, or a wireless mesh network. Each segment provides best effort data delivery service to the upper layer functions that may be required to provide this service (e.g., routing, error control, congestion control, etc) are internal to the segment and hidden from higher layers. On top of segments is the transfer layer, which supports end-to-end data transfers overs multiple segments, similar to how IP supports connectivity in today s Internet. The transfer service runs on both the end-points and network elements, called Access Points (TAPs), that interconnect segments. In Tapa, the transfer and higher layers work at the granularity of application data units (ADUs) [13] rather than byte streams or packets. The use of ADUs and the typically small number of segments in an end-to-end path allow transfer layer to implement various multi-path and content-centric optimizations [33]. It also simplifies the deployment on in-network services, making it easy to leverage advances in technology (e.g., cheap storage). In this paper, we present three such diverse transfer services as case studies to illustrate the generality of our approach: i) Swift improves performance for mobile and wireless users by intelligently managing the available segments, ii) Catnap [17] provides energy savings to mobile devices by implementing a traffic shaping service at the TAP, and iii) Vigilante [2] provides a high performance content distribution service within an online social community. Finally, the transport layer implements specific end-to-end application semantics over the transfer layer. The presence of the segment layer makes the transport layer lightweight, isolating it from the details of specific network technologies and thus making it easier to support heterogeneity in the system. The introduction of segments, combined with the use of ADUs, means that it is easier to implement transport protocols with diverse semantics required by different types of applications (e.g., fully reliable, streaming applications, etc.) It also becomes easier to insert services into the end-toend path, while maintaining specific semantics between the end-points and the (possibly third party) network service. The key contributions of this paper include: i) the design of the Tapa architecture; ii) an implementation and evaluation of the core Tapa functionality and a transfer service designed

2 for mobile and wireless users (Swift); and iii) case studies of transfer services (Swift, Catnap and Vigilante) and how they can leverage the Tapa architecture in different ways. In the next two sections, we elaborate on the Tapa architecture and the key requirements it satisfies. In Section 4, we present the design of a basic Tapa prototype that implements the core architectural features. In Section 5, we present case studies on how diverse transfer services can be implemented in the Tapa architecture. Section 6 presents an evaluation of Tapa from two different perspectives: flexibility and performance. Finally, we discuss related work and conclude. 2. REQUIREMENTS We discuss four key network requirements that are important for accommodating diversity at the edge. 1. Accommodating Diverse Networks and Devices The original ARPANET was designed to inter-connect heterogeneous networks and hosts. However, by today s standards, they were in fact fairly homogeneous. For example, it was assumed that links would provide reasonable reliability (< 1% loss rate) [12] and that all end-hosts would be powerful enough to support the full TCP/IP stack. The networks that make up the Internet now are very diverse, ranging from very high speed optical backbones, to low speed, unreliable wireless networks that have very different properties from wired links (e.g., higher error rates, variable bandwidth and delay), causing problems for end-to-end protocols such as TCP [7, 19]. Similarly, devices, such as sensors and mobile nodes, are highly resource constrained, compared with traditional endpoints. Dealing with heterogeneity of devices and networks affects how we distribute functions across devices. For example, when applying the end-to-end argument [30], we can no longer view the communication subsystem as a single homogeneous module. As a result, while customized transport solutions have been developed for specific networks (e.g., mesh networks [26], vehicular networks[19], sensor networks [27]), deploying them in today s Internet is difficult. One approach is to use different transport regimes for wired and wireless networks, with a proxy in the middle (e.g., I-TCP [7], PEP [11]). However, in the current architecture these solutions break the end-to-end semantics and they also interact poorly with many optimizations (e.g., hand-offs due to mobility or use of multiple interfaces/aps). 2. Accommodating network services Most applications in today s Internet benefit from rich network services, e.g., web caching, video transcoding, services for mobile and wireless users, etc [18, 29]. Unfortunately, inserting services in an end-to-end path is hard in today s Internet for two reasons. First, the data plane of existing protocols (IP and TCP) works with packets and bytes streams, which is too fine a granularity for most in-network services. Second, and more importantly, TCP s end-to-end semantics are very rigid they do not allow role of an intermediary or accommodate different delivery semantics. As a result, network services are typically hidden from the end-points (e.g., transparent web proxies) or deployed as overlays. This is not ideal, since there can be poor interactions with other services, like firewalls, and sharing services between applications, users and providers is hard. Earlier proposals, like DOA [35] and NUTSS [22], deal with network level middleboxes, such as NATs and firewalls, while our focus is on flow middleboxes [21] that provide higher level services, such as caching. 3. Relaxed End-Point Synchronization Current Internet protocols assume that the communicating endpoints will be simultaneously connected to the network, and key mechanisms for reliability and congestion control implicitly rely on this [12]. However, this is increasingly no longer the case. While DTNs [20] have recently received a lot of attention, intermittent connectivity has been an important research topic in mobile computing for a long time [31]. Relaxing the endpoints synchronization requirements is however not only important for wireless devices as increased interest in energy efficiency means that wired hosts may also be offline for longer periods of time, making relaxed synchronization an issue for the entire network infrastructure [4]. Clearly, the architecture must handle these disruptions as a routine matter by providing suitable mechanisms within the network, as well as appropriate end-to-end transport semantics to the applications for dealing with various types of disconnections. 4. Improved Topology Control The network path chosen for data transport is traditionally chosen by the network layer, but researchers have identified many cases where it would be useful to apply higher level criteria, such as identifying disjoint paths (for availability) or considering service availability, during path selection [5]. While BGP does provide policy interfaces, they are limited and typically not accessible to users. Topology control is especially important for wireless devices as they often have a choice of access points they can associate with or have multiple interfaces that use different technologies [24, 34]. This leads to a rich set of connectivity options, and the choice will often be based on high level information such as service agreements, service availability, mobility history etc. System Concepts The above requirements are very broad, but we can identify two underlying themes. The first theme suggests that we should raise the level of abstraction that the network provides to the higher layers (e.g., transport) of the system, since this may make it easier to hide diversity at the lower layers, e.g., allowing the deployment of custom network protocols. Raising the level of abstraction affects not only the protocol function, but also addressing, unit of data transfer, etc. A second concept, decoupling network regions with very different properties, and as a result making the regions fairly homogeneous, can help in a number of ways. First, it can simplify the deployment of end-to-end solutions over homogeneous networks. Second, the nodes that implement decou- 2

3 Segment Path (Wired) Path (End - TAP - End) Segment Path (Wireless) Transport Path (End-to-End) Tapa TCP/IP Stack Fixed Host Transport Segment Internet Segment Segment TAP Mobile Host Transport Segment TAP Semantics Optional Funcs: Multipath etc Segment Mgmt ADU Forwarding Error Control Routing Congestion Control... Semantics E2E Conn Mgmt Congestion Control Flow Control Routing Packet Forwarding Error Control TCP IP Link b) End-to-End Communication in Tapa a) Comparison of Protocol Stacks Figure 1: High Level Overview of Tapa pling can be used to insert additional functions (e.g., network services, storage for caching or relaxed end-point synchronization). Finally, by making the points of decoupling visible to applications, it gives applications some control over topology and to implement specific semantics between the end-points and in-network services. Note that we can view the abstraction and decoupling concepts as affecting modularity in the vertical (across layers) and horizontal (across the network topology) dimensions of the system. In the next section, we build on these two concepts to introduce the Tapa architecture. ïż 3. TAPA:ARCHITECTURE We first give a high level overview of Tapa, focusing on how we distribute functions between end-hosts and TAPs. We then elaborate on the key segment, transfer, and transport functions. 3.1 Overview Figure 1(a) compares the Tapa protocol stack with the current TCP/IP protocol stack, showing how Tapa unbundles the transport layer of the current Internet. We illustrate the role of the new layers in Tapa using an end-to-end communication involving a mobile client, one or more TAPs, and a fixedhost (Figure 1(b)). In this case, an end-to-end path consists of two segments, wired and wireless, that are joined by a TAP. Segments can be very diverse, ranging from various wireless access networks (e.g., multi-hop mesh networks, blue-tooth, etc), to wired segments (e.g., paths in the Internet or an enterprise network). Each segment is responsible for delivering data from one end of the segment to the other, employing congestion control, routing, and reliability mechanisms that are appropriate for that particular segment. The segments in an end-to-end path are managed by the transfer layer, which spans the client, TAP, and the fixedhost. TAPs provide the glue required to combine multiple segments (e.g., buffer space) and also have sufficient storage that facilitate relaxed synchronization between segments and end-points. TAPs also actively collaborate with the client or with other TAPs, sharing information about the services they offer as well as their capabilities. An endto-end path between a client and server can use multiple paths, either simultaneously or over time, as suggested in Figure 1(b). This adds a topological element to the transfer layer, offering opportunities for optimizations such as multipath and content discovery. Similar to IP in today s Internet, the segment and transfer layers offer a best effort service to the transfer and transport layer, respectively. This means that they deliver data with high probability, but delivery is not guaranteed. The motivation for the best effort nature of the segment and transfer services is the same as for IP [30], i.e. full network-level reliability is expensive and not always needed. Therefore, Tapa s transport layer, which is implemented on top of the transfer layer, still provides reliability and other application semantics to the end-points, albeit over a very short path consisting of a small number of segments. For example, a reliable data streaming service would require end-to-end acknowledgements, data ordering, etc. The transport layer can also accommodate the insertion of services on the TAPs, as we discuss in Section 3.4. Finally, the transfer and transport layers use ADUs [13], rather than byte streams or datagrams, offering a higher level of abstraction while enabling a variety of optimizations (Section 3.3). Tapa can be implemented as a native architecture, or as an overlay, although as we explain in Section 7 most of its features (e.g., asymmetric roles for TAPs and end-points) are unique compared to traditional overlays. 3.2 Segment Layer The segment service is responsible for transferring ADUs across a segment, e.g., client - TAP or TAP - server. Segments can choose the data granularity they use internally (e.g., frames, bytes, etc.) allowing them to optimize communication as appropriate. Segment endpoints do not move, so they can use network-specific locators as addresses. For example, in the wired Internet IP addresses based on CIDR may provide the necessary scalability, while MAC addresses may be more appropriate in wireless networks. As Tapa targets diverse edge scenarios, the lifetime of a segment may be very short lived (e.g., mobile clients), or they can be long lived (e.g., wired segment). 3

4 Segments must implement error, flow, and congestion control as needed. Tapa s transfer service expects segments to be reasonably reliable, but segments can use very different ways for achieving that, e.g. TCP style retransmissions and ACKs over wired segments versus in-network coding or opportunistic forwarding and block acknowledgements on a wireless segment [26, 9]. It is similarly well known that congestion control mechanisms need to be tailored for different types of networks [7, 26]. Some segments may need specialized routing protocols for delivering data over multiple hops within a segment (e.g., wired Internet or mesh network). Some segments (single hop, point to point) may not require routing or congestion control. The important point to note is that all these functions are implemented inside a segment and hidden from upper layers. Server Transp TCP IP Data Link PHY Transp TCP IP Case 1 Case 2 TCP IP Data Link PHY TCP IP TAP Bluetooth PHY HOP IP Mobile Client Transp Bluetooth PHY Transp Tapa Xfer HOP Figure 2: Some sample Tapa configuration cases. Segments can be very diverse and customized for each environment. Figure 2 shows two example segment implementations. If we have a mesh access network then we can use a segment protocol like HOP [26], which relies on per-hop mechanisms for reliability and rate control. In contrast, we can use existing end-to-end solutions for the wired segment. Segments can also bypass IP and traditional link layers, using options as diverse as blue-tooth and wireless USB. This is a direct benefit of raising the level of abstraction not all networks and devices need to use TCP/IP in order to be part of the Internet. In Tapa, mobile clients can run light-weight customized protocol stacks for the segment technologies they use to communicate with a TAP, without worrying about heavy-weight functions that are required to communicate over the Internet. This will lower the barrier for next-generation of embedded devices and sensors to communicate over the Internet. Optimizations: We expect most segments to provide a fast service i.e., deliver data as fast as possible similar to how TCP works in today s Internet. However, in some cases, applications may want to use a slow segment for transferring delay tolerant data, thereby reducing their peak network bandwidth usage. Examples of such transfers include a large volume of communication between data centers (for backup purposes) or for pre-fetching data for later use by the client (e.g.,vigilante). In today s Internet, using such background IP transfer protocols forces end-points to remain up for longer duration of time, consuming end-point resources. In Tapa, an end-point can delegate this responsibility of staying-up to the TAP for example, a mobile client can send data using a fast segment to the TAP who can use a slow segment to transfer data to the server. In order to properly select segments, the higher layers must be able to determine key properties of a segment (e.g., latency, available bandwidth, etc). In the case studies, we show how such information can be used by the transfer layer to optimize ADU transfers. 3.3 Layer The transfer layer is the inter-networking layer of Tapa, managing end-to-end ADU transfers over multiple segments. Its role is somewhat similar to that of IP in today s Internet: delivering data over multi-hop paths. However, there are several differences in both the control and data plane. Internet routing needs to establish routes in large scale but fairly stable networks; in contrast, the Tapa transfer layer establishes short (e.g. two-segment) paths but paths can be very volatile due to the dynamics of the access network (e.g., mobility and wireless dynamics). IP packet forwarding is optimized for high throughput despite large forwarding tables; ADU forwarding in Tapa is simple, but needs to accommodate in-network services (e.g., transcoding, Catnap). The introduction of the transfer layer is in part to allow for a separation of concerns. The segment layer can focus on the challenges associated with specific networks (e.g. scalability within core Internet) while the transfer layer can focus on dealing with higher level challenges (e.g. selection of segments based on content or service availability, mobility). We describe the basic working of the control and data planes, highlighting the important functions they need to perform. In Section 5, we describe how we can implement diverse transfer services at the transfer layer both at the data plane as well as the control plane Control Plane The control plane of the transfer layer deals with establishing segments to set up an end-to-end path, enabling the data plane to transfer ADUs over them. In order to set up the segments, the transfer layer needs the globally unique identifiers for both end-points and TAP(s). The segment layer must be able to translate these identifiers into locators that can be used to establish the segments; locators may only have meaning locally within a network, i.e. they do not have to be globally unique. Our prototype uses host-names as identifiers for convenience, but self-certifying identifiers would be more appropriate for an actual deployment[6]. The identifiers are mapped to locators using either a naming service service (e.g., DNS for wired Internet segments), or based on a locally maintained mapping to internal Addresses (e.g., MAC addresses for a bluetooth based segment). In Tapa, the end-point s identifier is specified by the application, similar to how applications manage end-to-end 4

5 communication in today s Internet. The transfer layer has the option of creating a direct segment with the other endpoint, or it can leverage TAPs, in which case, it must be able to discover TAPs (i.e. the identifiers) that may be useful in assisting with the communication. The case studies show how TAP discovery can be based on both local neighbor discovery and application-level information. The transfer layer also needs to do congestion control over the multi-segment path to ensure that TAP buffers do not overflow. This task is much simpler compared to congestion control over the entire end-to-end path (including the wired Internet and the access network) as a transfer-level path in Tapa typically only involves a limited number of segments. Note that congestion control in Tapa should be interpreted more broadly than in, e.g. TCP, since relaxed synchronization, segment hand-off, and multi-path operation can create discontinuities in the network path that can interact poorly with congestion control. Several congestion control solutions could be considered, including end-to-end TCP-style based on implicit feedback, ECN-style solutions based on explicit feedback, and back pressure based on segment-per-segment flow control [26]. Since transfer-level congestion control operates at a higher level in the stack (e.g., on ADUs) and at the edge of the network, we decided to use the last option in the Tapa prototype since it fits well with the segment model, but other solutions should be evaluated as future research Data Plane The data plane deals with transfer of ADUs [13] over multiple segments. ADUs offer a middle ground between existing applications that generally use containers (e.g., file names) and IP, which uses datagrams. In Tapa, ADUs can be defined in a flexible manner based on the requirements of the application. For example, an ADU could correspond to a whole file (e.g., Catnap[17]), to chunks within a file (e.g., DOT[33]), an MPEG frame in a video transfer, or a single command in a telnet session. The use of ADUs in Tapa is important as they are a natural unit for exploiting multiple TAPs or multiple interfaces to download a file, or for inserting services such as caching in the end-to-end path. ADUs must be identified using a label. In our prototype, we use content-based hashing [33] to name data i.e., we use self-certified labels to identify ADUs. This allows the network to serve the ADU request in an opportunistic manner: TAP can serve the ADU from its local cache or query one-hop neighbors (e.g., another TAP that acts as a cache within an enterprise/isp network or other TAPs within a social network). Similarly, it can use a routing protocol that can provide content reachability at the Internet scale level, if this becomes feasible in future. Our current prototype design takes a pragmatic but flexible approach towards content centric networking. Similar to DOT [33], it allows applications to provide hints to the TAP; these hints correspond to hosts that are guaranteed to have the data (e.g., a web server), so if TAPs cannot route on ADU identifiers they can use the hint to retrieve the ADUs. The use of ADUs changes the interface between the transport layer and applications, compared with the socket API. lications should be able to specify the associated requirements with each ADU such as, importance of delivery, limit of the usefulness of the data, etc. 3.4 Transport Layer Tapa s transport functionality is very thin as it operates over a limited number of segments and most of the heavy lifting performed in traditional transport protocols is already implemented within the segment layer. As an example, error control is largely handled in the segment layer, and the role of the transport layer is to deal with lost ADUs, e.g. as a result of TAP failures or non-recovered ADUs during handoff for mobile users. Similarly, the transport layer may need to reorder ADUs that were delivered out of order, e.g. as a result of the use of multiple segments. Important thing to note is that we can easily implement support for different semantics (e.g., partial reliability or out-of-order delivery). Basic functions like reliability, and ordering will typically involve only the end-points, but some semantics may also involve network services running on one or more TAPs. In Tapa, TAPs are visible to the end-points, opening the door for richer semantics that explicitly capture the role of services running on the TAP. This is important because connections increasingly involve middleboxes that affect end-to-end semantics [11, 18]. Unfortunately, protocols such as TCP, cannot accommodate intermediaries, requiring applications to implement this functionality at the application level, using separate TCP connections with the third-party and between end-points. Here are some examples of functions that can benefit from interaction between the end-points and the network: Caching: Web and other types of caching within the network is already widely used and is typically supported at the application level (e.g., HTTP). Data Integrity: Most applications rely on end-to-end integrity check (e.g., CRC, hashes, etc) for their correct operation, but some network services may change the bit streams in the middle of the end-to-end path (e.g., transcoding). Authentication: Trust is becoming increasingly more important for users and applications. At times they may have more trust on an end-point (e.g., a bank) and in some cases they may trust an in-network element (e.g., a TAP owned by your enterprise) more than the end-point. Confidentiality: Data confidentiality is important but is often at odds with the need for in-network elements to see the data contents [10]. The most practical approach is controlled transparency, allowing explicit involvement of network elements with a flexible degree of control [14]. Tapa can deal with these functions as part of of transport, using a single session involving the TAP and the end-points. Let us use some example scenarios to illustrate this flexibility: 5

6 lication Independent Caching: Imagine a user who is interested in downloading a file or a video and the content is already available in the cache of a TAP. In some cases the client may not trust the TAP (e.g., open wifi), so the client transport will request integrity check from the endpoint while the TAP can serve the data in an application independent way (similar to DOT [33]. Video Streaming: Imagine a client streaming a video on her resource constrained mobile phone, requiring a low resolution version of the video. The TAP provides the required video transcoding which means that data integrity checks can only be on a per-segment basis (Server - TAP, and TAP - Client). However, client may still require authorization from the server to get the video from the TAP. Download from un-trusted Server: Imagine an enterprise client downloading software from an untrusted web server. The client will only run the software if it is verified by the enterprise TAP, which employs the latest virus and worm scanning techniques. In this case, there is no direct contact between the client and server transport (they only talk to the TAP). The transport of the TAP will first finish the whole download, perform the checks, and then send the data to the client which will make sure that the download is verified by the TAP (through authentication and integrity between client and TAP). Secure Chat Session: In the normal case, transport of only the two end-points is involved with no transport layer role of the TAP. The session will provide end-to-end authentication and confidentiality. However, if a third party (e.g., government) wishes to observe all communication, they can make it mandatory for end-points to provide explicit control to a government TAP to observe the transfer. The transport layer between end-points and TAP can ensure that only the government TAP (and not any other TAP) can observe the unencrypted data and any changes to the data, even by the government TAP, is not allowed. So data integrity is endto-end; data is encrypted and can only be decrypted by the government TAP or the end-point. 3.5 Tapa and the end-to-end arguments The end-to-end (e2e) arguments have played a major role in the success of the Internet [30, 12]. Let us briefly comment on two important implications of the e2e arguments as they apply to Tapa. A first implication is that the network should remain simple since a lot of functionality needs to be implemented on end-points anyway. Even though Tapa appears to push more functionality into the network, it is in fact consistent with this argument. For example, by using segments to decouple heterogeneous parts of a network, we can push functions to the segment end-points, if appropriate, while heterogeneity would make it hard to do this for the end-to-end path. As another example, supporting network services on TAPs does not push functionality into the network (i.e., communication subsystem), but it rather offers a way of providing systematic support for an essential network requirement. The second important implication is transparency of the network what goes in, comes out. Tapa embraces the reality that the presence of various players within the network compromises transparency [10]. As advocated by Clark et al. [14] the best way of dealing with this is to provide some form of controlled transparency, which Tapa provides (e.g., control over level of confidentiality, etc) as part of the transport semantics. 4. PROTOTYPE DESIGN We have designed and implemented a basic prototype of Tapa with the goal of supporting the core features of the architecture. The prototype enables transfers of ADUs between a client and server through one or more TAPs. Our design decisions are based on the assumption that TAPs are being used in typical home wireless access scenarios. Broadly, there are two modes of transferring ADUs a pull mode and a push mode. In the pull mode, applications use the get API (Figure 3) to retrieve an ADU; the call specifies the ADU identifier along with a hint that refers to a server that has the ADU. The push mode is used to send an ADU to a particular node; it is useful for sending control messages between the client or server (e.g., acks) or for dynamic content (e.g.,voice) that is a poor fit for the pull model. We now describe the role of the three layers in enabling an end-to-end data transfer between a client and server through a TAP. 4.1 Segments An end-to-end path consist of two segments: one between the client and TAP, and the other between the TAP and server. We use different segment protocols based on the properties of the underlying segment. Our prototype uses TCP/IP as the segment protocol on the wired side, but we reuse TCP segments across ADU transfers to minimize the state keeping required at the TAP as well as to improve per-segment congestion control. Ideally, Tapa will benefit from a lightweight alternative to TCP for the wired segment, especially if TAPs are used in enterprises or core networks. Our prototype also supports various wireless specific segment protocols: Blast, a protocol that is optimized for sending data on typical networks; HOP [26], a protocol designed for multi-hop mesh networks and a Bluetooth based segment protocol. Most of the segments used in our case studies are fast segments that transmit data immediately, but Vigilante can also use a background segment. 4.2 Layer In the core Tapa design, the control plane of the transfer layer is simple and just needs to set up the segments to transfer ADUs. However, as we show in case studies, the control plane can also play a pro-active role during data transfers (e.g., in case of mobility). The client selects the segment protocol for each segment as well as the segment end-points i.e., TAP for the client-tap segment, and a server for the 6

7 Function Description lication API getadu(id(s), requirements, source-hint, source-app, callback) Call used to pull an ADU sendadu(id(s), data, requirements, destination, dest-app, callback) Call used to push an ADU register(appid, callback) registration for receiving pushed ADUs Segment Layer API register(callback ) Registering to receive incoming Tapa msgs sendmessage(tap id, message, reqs, callback) sending a message across the segment Figure 3: Tapa interfaces to the lication and Segment Layers. Interface between transfer and transport uses a subset of these APIs. TAP-server segment. The client usually uses a default TAP for all communication, but in the case studies, we show how we can support advanced TAP discovery and probing mechanisms. The other end-point i.e., the server, is specified by the application as a hint. Once segments are set up, the data plane takes over the responsibility of transferring ADUs and delivering them to the higher layer. The default data plane at the TAP simply reads from one segment and immediately writes to the other. However, other transfer services can be used. In Catnap, for example, the TAP batches data and then transfer it over the wireless segment, thereby allowing the mobile client to sleep between burst transfers. Once the transfer layer assembles the whole ADU it sends it to the transport layer. In case of TAP, this is optional and only required if the ADU needs to be cached or delivered to an upper layer service. Headers: The transfer layer header consists of (src-id, dst-id, TAP-id(s), app-id) while the ADU header consists of: (id, length, offset, type) Most fields are quite standard and self explanatory; type corresponds to whether it is an ADU request or a response. 4.3 Transport Tapa s transport is thin and implements three types of functions: reliability, ordering and delegation semantics (earlier discussed in Section 3.4). We currently only provide support for caching, as part of the delegation semantics, but applications can specify a variety of roles of how the cache at a TAP can be used (e.g., to offload transfer responsibility from a client or from the server, etc); this is used extensively in the Vigilante case study. In the case studies, we elaborate on how we can use different types of acknowledgements to implement various application semantics involving the end-points and the TAP. 5. CASE STUDIES We now present three case studies that show how Tapa s basic design can be extended to implement rich optimizations as transfer services. 5.1 Swift Swift addresses the challenges and opportunities specific to wireless and mobile environment by implementing various optimizations as part of a transfer service that runs on the client: it can quickly switch to an alternate segment, retrieve ADUs cached at a nearby old TAP in case of mobility, and multiplex multiple segments using a single network interface (e.g., FatVAP [24]) or by combining multiple interfaces to improve throughput. We discuss more details about these optimizations, including some performance results, in Section 6. We explain how the three Tapa layers behave in Swift with the help of a scenario that involves a client downloading ADU(s) from a server via a TAP. We assume that the client already knows the identifiers of the ADUs Segments Swift can use any segment protocols available in Tapa and the internal working of the segments does not change. Swift s novelty comes from how segments are used by the transfer layer to optimize user performance. The use of separate wired and wireless segment protocols decouples these two diverse segments, thereby shielding the wired segment from the vagaries of the wireless segment. As Swift targets highly dynamic scenarios, segment protocols that have a low set up overhead provide better performance. Also, another useful function is to provide fine grained information regarding segments in the form of their availability and their properties (e.g., expected performance) to the transfer layer. We have implemented a module for based segments that provides information about the presence of a TAP and the expected performance based on beacons and received signal strength information (RSSI), respectively Layer The data plane of Swift is the same as the one used in the basic Tapa prototype. All the intelligence of Swift lies in the control plane, which is responsible for three things. 1. TAP discovery and probing: This involves identifying TAPs that can be used for data transfer as well as the segment properties to these TAPs. Swift uses information from the underlying segment protocol for this purpose (described earlier), although future implementations can support more advanced discovery and probing mechanisms (e.g., mobility information for TAP discovery and bandwidth estimation techniques for probing). 2. Managing Segments: Upon receiving a request to 7

8 retrieve an ADU, and assuming no segments are available, the control plane needs to establish segments to transfer the ADU. Subsequently, it also needs to monitor progress as well as the availability of TAPs to adjust segments as needed. The initial segment selection is based on the following information: ADU requirements, spatial resources available (TAPs and services they provide), and the connectivity to the available TAPs. For example, the properties of a 3G interface in terms of throughput, latency, reliability, and cost, are very different from Wifi. In the general case, the decision outcome comprises a suitable TAP and interface. Once the transfer starts, the control plane needs to monitor and adapt. There are two aspects to monitoring. First, the data plane reports the status of ADU transfers, providing information about future transfer needs (i.e., identifiers of missing ADUs). Second, the control plane collects information about the status of existing segments and opportunities to use new TAPs and segments, based on the information from TAP discovery and probing process. Based on this monitoring, the control plane decides whether it needs to take any action, such as switching to a new TAP. In case of mobility, this may be required while in other scenarios it may be beneficial for performance reasons. A segment is set up with the new TAP and requests for missing ADUs (as indicated by the data plane) are sent over this segment to the old TAP, or to the server if needed. We discuss TAP failures below. It is possible during a transfer that the TAP discovery process detects another segment that can be used in parallel with the existing segment. Examples of such scenarios include residential wireless scenarios where the up-link bandwidth of APs can be aggregated or the simultaneous use of multiple interfaces [24, 34]. If such an opportunity is available, the control plane will establish another segment and will coordinate with the data plane on how to stripe the ADUs over the available segments Error Recovery: The transfer layer can encounter a variety of errors. The loss of segments due to mobility or changes in the wireless network are handled as a routine event, as described above. TAP failures are more challenging for two reasons. First, there is no warning that the segment will disappear, so it is more difficult to transition to a new segment, and connectivity disruption is more likely. Second, the state on the TAP is lost. The transfer layer is based on soft state, so any ADUs stored on the TAP will have to be retrieved from the source. Recovery of ADUs that are lost due to a TAP failure or handoff involves both the control and data plane. The data plane reports the missing ADU ids and the control plane retrieves it from the server or from the old TAP (in case of mobility). 1 Our system evenly distributes the ADU requests if multiple TAPs with similar performance are available. Figure 4: Basic Idea of Catnap. The slow wired segment is active for the whole duration while the wireless segment only remains active for a small duration at the end Transport Swift uses the standard transport functions provided in Tapa: reliability, ordering and delegation semantics. In the above scenario, the server can delegate the responsibility of delivering ADUs to the TAP, thereby relieving itself of the burden of repeatedly retransmitting ADUs due to client mobility or poor wireless segments. This means that even if a client moves to a new TAP, it can retrieve the content from the old TAP who has been delegated the responsibility by the server. However, application semantics will typically still require that the server is notified when the client receives the ADU, as application functions may rely on this information. Such semantics can be supported through the use of an end-to-end ack between the client and server at the end of the transfer. This means that the server application can specify that it is interested in acks from both the TAP and the client, and it is the responsibility of the transport to ensure that these acks are delivered and reported to the application. 5.2 Catnap Catnap allows a mobile client to sleep during ADU transfers by intelligently shaping when data is sent on the wireless segment. Catnap targets settings where the wireless segment is much faster compared to the wired segment. For example, as shown in Figure 4, the wired segment, which is the bottleneck, remains active for the duration of the transfer while the much faster wireless segment is inactive most of the time, although it still completes the transfer on time. During the time the wireless segment is inactive, the mobile device can enter into various sleep modes (e.g., Power Save Mode, Suspend-to-RAM mode, etc), thereby providing significant energy savings to the mobile client [17]. Catnap leverages the key concepts of Tapa in the following ways. First, relaxed synchronization between end-points, provided through the use of different segment protocols and buffering at the TAP, allows the wireless segment to be inactive even though data continue to flow on the wired segment. This is not possible with end-to-end TCP that relies on strict synchronization between end-points. Second, the concept of ADU allows the TAP to know when data will be 8

9 consumed by the client application. So, initial packets of an ADU can be delayed as long as the finish time of the ADU remains the same. Third, fine-grained segment information regarding performance of both wired and wireless segments is used in order to ensure that transfers are scheduled at the right time, which provides maximum sleep opportunities without increasing the end-to-end transfer times. Catnap s functionality can be implemented as a transfer service that runs at the TAP. We explain how the three different layers work in Catnap with the help of a scenario that involves a client downloading ADU(s) from a server via a TAP Segments Catnap can use the various segment protocols available in Tapa. Again, Catnap does not change the internal working of the segments. Instead, Catnap s transfer layer simply controls when data is transmitted on these segments. Since it attempts to send a large amount of data as fast as possible, segment protocols that are optimized for bulk transfers can provide additional advantages. Finally, Catnap requires accurate estimates of the performance of the wired and wireless segments, so it uses passive bandwidth estimation techniques that are specifically tailored for wired and wireless segments [17] Layer The control plane performs the task of setting up segments and making requests for ADU transfers, while the data plane implements Catnap s traffic shaping services that decides when data is transmitted on the wireless segment. As mentioned earlier, it needs to probe the underlying segments to get information about the available bandwidth. The bandwidth estimates, combined with the size of the ADU, determines when transmission is scheduled. Moreover, segment conditions can change, especially in a wireless setting, so the transfer service needs to constantly monitor the segment properties and adapt the scheduling decision based on the changes Transport The delegation semantics are used to support flexible mobile scenarios that are possible with the help of Catnap. For example, a mobile client can start a large download (e.g., a movie) in the morning, go offline, and retrieve it from the TAP in the evening. Similarly, in order to upload a large file to a server, the mobile client can burst the data to the TAP, go offline, and the TAP can take over the responsibility of sending data to the server over the slow wired link. In both scenarios, applications may be interested in differentiating between when the data is received by the TAP compared to when it is received by the end-points. For example, a photo upload application may like to hold on to the data as long as it is not received by the server. Using Tapa, applications can specify whether they are interested in an ack from the TAP, end-point or both, allowing them to implement different kinds of semantics. 5.3 Vigilante In another system, we have implemented a photo sharing service, Vigilante, that leverages TAPs to store and distribute content within an online social community [2]. Here we focus on how Vigilante benefits from an architecture like Tapa. The key motivation behind Vigilante is that friends are usually located close-by, so it is faster to retrieve content from a friend s TAP rather than a remote server. Moreover, a user s access patterns and preferences are well defined, so there are opportunities to pre-fetch data at the TAP. Finally, since a typical user has a limited number of friends, the content centric transfer optimizations are easy to implement since scalability is not a concern. Vigilante leverage s Tapa support for content centric optimizations: lications specify the identifiers of ADUs of interest, along with hints identifying the TAPs of friends that are likely to have a copy of the data. These hints can be easily generated based on the user s social network information. The TAP then retrieves the ADU from the best possible option within the social community. So Vigilante heavily relies on a content centric network amongst TAPs whereas Catnap and Vigilante provide optimizations within an end-to-end path involving the client, TAP and server. Also, unlike Swift and Catnap, where the role of the application is limited to providing an identifier for the server as a hint, the role of the application is much more important in Vigilante (e.g., they provide information about other TAPs within the social network). We explain how Vigilante works by describing the role of each of the three layers Segments Besides the default fast segment type, Vigilante can also use a rate limited/slow segment for transferring data that is not immediately required by the destination. This can be used both when pre-fetching data onto a TAP, or when data is transferred to a server for backup purposes Layer Vigilante is implemented as a transfer service that runs on the TAP. The additional support of Vigilante is part of the control plane as it needs to make the decision on which segments to use for data transfers. lications use information about the social relations and access patterns of a user to help the control plane in making these decisions. For example, TAP discovery in Vigilante is based on information from the social networking application regarding TAPs of the user s friends (they provide the TAP identifiers). These TAPs form a content centric network, exchanging content information amongst them. For example, they can exchange the ADU ids they have in their cache or the bloom filter corresponding 9

10 to these ids. We can also have a centralized approach where all TAPs report their cached ADUs to a server who can then pass this information as hints to the client application. Due to the limited size of a social community both options are feasible. Vigilante currently supports the latter option. lications request content by specifying the ADU id and the corresponding hints to the lower layers. The transfer service at the TAP retrieves the content from the best possible TAP, whose selection is based on information about segment properties (i.e., available bandwidth and latency). After retrieving the ADU, the TAP can cache the ADU. As mentioned earlier, TAPs report the newly cached ADU id to a central server, who then updates the hints. As a result, a social networking user who connects to the central server and asks for a content is sent the latest hints, which correspond to all the TAPs who have the content in their cache. In order to publish an ADU, the application sends it to the TAP, who stores the ADU in its cache. Naturally, other users in the social network need to know about the newly published ADU this is done at the application level by notifying a central server, although alternate approaches where TAPs can directly communicate this information amongst them, in a purely distributed manner, can also be considered. 2 Finally, at the time of publishing the data, applications can also preload the TAPs of the user s friends creating multiple copies of the ADU ensures better performance and fault tolerance as there are multiple options to retrieve the content Transport lications can also implement various semantics involving the TAP and server in the data transfer. If the server also participates in distributing content then applications can choose a TCP based segment to transfer the ADU from the TAP to the server. However, if the server only maintains a backup of the data then applications can choose a slow segment to transfer data from the TAP to the server. Depending on the role of the TAP, applications can also specify whether they are interested in an ack from the TAP, server or both, ensuring correct application behavior under these various scenarios. 6. EVALUATION The primary goal of our evaluation is to show the flexibility of Tapa under diverse scenarios. Unfortunately, there is no single agreed upon metric that can quantify flexibility. Therefore, our evaluation approaches the question of flexibility from two different angles. First, we show how Tapa can support diversity at multiple levels: applications, segment layers, and transfer optimizations. We consider the various optimizations supported in Swift our focus is not on performance but rather on showing flexibility by demonstrating functionality as well as quantifying the effort in adding that functionality (lines of code and man hours required). In the 2 This requires application specific support at the TAPs Mbps 40 msec MH TAP FH MH Emulated wireless i) Single-WAN-EMU 3Mbps 40 ms iii) Multi-WAN-EMU FH MH Real wireless 100Mbps 3 msec MH TAP FH ii) Single-LAN-EMU 6Mbps 30 ms v) Multi-WAN-CMU Figure 5: Various Topologies used in the Evaluation. Ones with Emu suffix correspond to Emulab testbed while those with CMU correspond to CMU wireless emulator. Wired links are emulated. Type Remarks Segments HOP, Bluetooth, TCP, Blast Swift Optimizations Multiplexing Multiple Segments, Caching (On-Path and Off-Path) lications New and Legacy (Firefox) Table 1: Diverse options supported by Tapa prototype. second part of the evaluation, we use micro-benchmarks to quantify the overhead of Tapa. Experimental Setup: We have used both a real world and an emulation-based testbed for our evaluation. (Figure 5 shows the various topologies used in the experiments). We used b mode with default values. We used adhoc mode to avoid the switching overhead associated with managed mode. This is a practical alternative to the various solutions that allow fast switching between APs and recent studies have used a similar approach for such experiments [8]. Unless otherwise noted, we conduct average of five runs (deviation is within 10% of the mean). We varied the ADU size from 64kB to 1MB. 6.1 Accommodating Diverse Options Table 1 lists the various options supported by our prototype at different levels Supporting Diverse Segment Protocols A key goal of our prototype was to enable decoupling of wireless and wired segments so that diverse segment layers can be used on the wireless side. We now discuss our experience in adding different segment protocols to Tapa. TCP: We added support for TCP as a segment on both the wired and wireless side for two reasons. First, in some network scenarios (e.g., wired), it provides many of the features that we expect from a segment protocol. Second, as its use is well understood, it allowed us a convenient way to implement a working segment protocol, thereby simplifying the implementation and debugging of the prototype. HOP: HOP is a possible replacement for TCP in multi- FH 10