Title: Collaborative research: End-to-End Provisioned Optical Network Testbed for Large-Scale escience Applications

Transcription

1 Year 4 Activities report for the NSF project EIN Title: Collaborative research: End-to-End Provisioned Optical Network Testbed for Large-Scale escience Applications Date: July 21, 2007 PI: Malathi Veeraraghavan, mv5g@virginia.edu Please reiterate the goals and objectives of your efforts, and summarize the research and education activities you have engaged in that aim to achieve these objectives. Include experiments you have conducted, the simulations you have run, the collecting you have done, the observations you have made, the materials you have developed, and major presentations you have made about your efforts. In a later section you will list more formally any publications and other specific products (database, collections, software, inventions, etc.) that have resulted. Our goals and objectives were to extend the CHEETAH network, interconnect it to the HOPI testbed, develop applications that use dynamically shared circuits in a user-transparent manner, and advance our theoretical understanding of bandwidth-sharing modes in circuit-switched networks. Specifically, we focussed on applications that could benefit from the use of high-speed, dynamically shared, circuits held for short durations. The objectives of our educational activities were to support Masters and doctoral students in their research, and offer our undergraduate students opportunities to conduct experiments and implement software for the CHEETAH network. Our basic research activities consisted of developing analytical and simulation models for the Immediate-Request (IR) and Book-Ahead (BA) modes of bandwidth sharing. Our experimental research activities focussed on connecting the CHEETAH network to the HOPI testbed, and developing control-plane and application software for experiments across these testbeds. We demonstrated our control-plane software and an application (circuit-aware squid, which operates as a connectionless-circuit gateway) on the HOPI testbed at the July 07 Internet2 J.Techs meeting. Our educational activities consisted of graduating and/or advancing our graduate students through the various steps of their doctoral degree programs (e.g., qualifying examinations, proposal defense, dissertation defense), and guiding undergraduates in their senior-thesis projects. The PI gave an all-day tutorial on GMPLS at IEEE Globecom06, the slides for which were coauthored by all team members (posted on project web site, 1

2 Below is a list of our activities 1 for the period, Aug July 2007, which includes experiments we have conducted, and some observations we have made (more details on our observations are listed in the Findings document): 1. CHEETAH control-plane and application testing on the HOPI testbed (all): After obtaining the approval of the HOPI technical teams on Feb. 13, 07 (presentation on our project web site) to deploy and test CHEETAH control-plane software and applications on the HOPI testbed, we obtained the necessary logins on the HOPI PCs and Force10 E600 switches. We upgraded our CHEETAH controlplane software to support sub-gbps VLAN based virtual circuits, and added a Force10 E600 switch programming module (obtained from the DRAGON team). We deployed and tested this control-plane software on the HOPI testbed (for more details, see Item 2 below). Next, we tested our new Circuit- TCP (CTCP) implementation (for more details, see Item 3), and the BWdetail software program (for more details, see Item 4). We implemented a circuit-aware application gateway, which is squid software modified to include our CHEETAH signaling client and CTCP (for more details, see Item 5). Finally, we implemented and tested a software module called the virtualizer (for more details, see Item 6). We demonstrated all these software modules at the Internet2 J.Techs meeting on July 17, 2007 on the HOPI segment between Los Angeles and Seattle (presentation on our project web site). We report our conclusions from this experience in Item 1 of the Findings document. We wrote two documents that describe this activity, Proposal for experimentation on the HOPI testbed, Jan. 29, 2007, and A discussion of goals and control-plane implications for our HOPI experiments, May 5, 2007, cheetah/documents/hopi/mv-dragon-cheetah-ctl-plane.pdf. The latter compares the CHEETAH and DRAGON control-plane solutions. 2. Control-plane software (Xiangfei Zhu): In Fall 06, we modified the circuit-requestor program to add command-line arguments for call holding time and call renewal. This new program was tested on the CHEETAH network. In Spring 07, we designed a new version of the end-host signaling client and circuit-requestor program to support sub-gbps VLAN circuits, which we refer to as CHEETAH Client System Agent (CCSA), and an external control-plane engine for Force10 E600 switches, called the CHEETAH Control-Plane Module (CCPM). The CCPM implements OSPF-TE and RSVP-TE, handles incoming call requests for VLAN-based virtual circuits through the Force10 E600 switch. It is a fully distributed solution in which route computation, connection admission control (CAC), VLAN ID selection, and Force10 E600 switch programming, are implemented in the CCPM, deployment of 1. We identify the primary student responsible for each of the activities. The postdoctoral fellow and the PI participated as guides/designers for all the activities. 2

3 which is one per Force10 E600 switch. Our results are presented in Item 2 of the Findings document. The software is posted on our project web site. 3. Transport protocol (Mark McGinley and Helali Bhuiyan): We implemented a new version of Circuit- TCP (CTCP) by exploiting a feature of Linux , which allows for new congestion control modules to be plugged into the system. This helped us overcome a drawback with our previous CTCP implementation, a patch for the Linux kernel, which necessitated constant updates as new versions of the Linux kernel were released. The ability to select CTCP as the congestion control algorithm has been integrated in the circuit-aware application gateway described in Item 5, and the BWdetail software tool (see Item 4) used for data-plane testing of circuits. We carried out several experiments comparing CTCP and BIC-TCP over three types of circuits established across the CHEETAH network. The Sycamore SN16000 switches allow for (a) Layer-1 circuits, in which a GbE port is mapped to an equivalent- or lower-rate SONET virtually concatenated (VCAT) circuit, (b) Layer-2 circuits, which are strictly speaking virtual circuits, created by mapping a VLAN on a GbE port to a single SONET VCAT circuit (rate-limiting is automatically enforced by the hard limit of the VCAT circuit on the backend of the GbE interface card), and (c) multiplexed Layer-2 circuits in which multiple VLANs on a GbE port are mapped to the same SONET VCAT circuit. We also ran experiments comparing CTCP/BIC-TCP using our BWdetail program on the HOPI testbed. Our results are reported in Item 3 of the Findings document. Both versions of CTCP software are posted on our project web site. 4. escience application support (Mark McGinley): To support escience projects and specifically TSI, we implemented a tool called BWdetail to test the throughput across end-to-end circuits. To support high-bandwidth connectivity for individual collaborators, systems must be highly tuned. BWdetail is a tool to measure available bandwidth, and provide detailed information relating to the TCP stack without requiring kernel modification. BWdetail can help identify limitations at the end-system that cause inadequate performance, and can reveal the effects kernel parameter changes have on the TCP flow. In addition, BWdetail is useful for analysis of TCP congestion control algorithms, both for comparison of existing algorithms or development of new ones. It displays parameters such as cwnd (congestion window) and the number of unacknowledged packets. Our findings are reported in Item 4 of the Findings document. The BWdetail software is posted on our project web site, and has been sent to the Internet2 performance group. 5. Circuit-aware application gateway (CAG) design and implementation (Xiuduan Fang and Robert W. Gisiger): Recognizing the lower costs of high-speed circuit-switch interfaces relative to packetswitch interfaces, we explored the concept of deploying circuit-switched networks in only the core of 3

4 the Internet. We developed an internetworking architecture in which the core network consists of two subnetworks, (i) a high-speed dynamically shared circuit-switched subnetwork, and (ii) a low-speed packet-switched (IP routed) subnetwork, interconnected to connectionless packet-switched regional/ enterprise networks. Fig. 1 illustrates the architecture. Given that TCP is a transport-layer protocol designed for connectionless networks, Circuit-aware application gateways (CAGs), deployed at the edge of the core circuit-shared subnetwork terminate TCP connections from/to http clients and servers. Noting from netflow data collected on Abilene that http is the dominant application protocol, we focus our CAG implementation effort to serve this traffic. In our prototype implementation of a CAG, we Figure 1: Internetworking architecture to introduce dynamically shared circuit-switched/virtual-circuit networks into the core started with the squid web-proxy/caching software, and integrated our signaling circuit-requestor software and transport-layer CTCP software. To direct traffic toward these CAGs, for this first prototype, we use a proxy auto-configuration file to configure Web clients to choose the closest CAG as its proxy to reach distant domains that are known to be geographically closer to other CAGs. When an http client request arrives at the CAG, if a parent CAG across the circuit subnetwork is appropriate for the Web server being accessed, the CAG initiates call setup, while immediately serving the current request via the IP-router subnetwork. Once circuit setup is complete, any subsequent http requests that need to be routed from CAG-to-CAG are sent over the circuit. We are exploring more general techniques to route select flows toward the CAGs (instead of requiring users to configure their web clients) using policy-based routing and iptables filtering in the CAGs. Our conclusions are reported in Item 5 of the Findings document. The CAG software is posted on our project web site. 6. A virtualizer for the HOPI testbed (Mark McGinley): The purpose of the virtualizer is to allow the Force10 E600 switches of the HOPI testbed to be controlled by two different control-plane solutions, the original DRAGON-based solution deployed on HOPI and our new CHEETAH control-plane solution. More generally, we implemented the virtualizer as part of an effort to virtualize the HOPI testbed 4

5 in order to support multiple GMPLS control- and management-plane instances to encourage more networking researchers to use this testbed. The virtualizer presents a virtual representation of the Force10 E600 to each networking-researcher s software. This enables a control- or management-plane software module to operate in a normal fashion, while the virtualizer checks commands issued by the controlor management-plane software module to ensure that it manipulates only its allocated resources. The resource set allocated to each experimental software module consists of some subset of the switch s ports, VLAN IDs, and bandwidth. Our results are presented in Item 6 of the Findings document. The virtualizer software is posted on our project web site. 7. CHEETAH network maintenance (Tao Li): Tao supported the graduate students, undergraduate students, Zhaoming Li, CUNY, Andrew Brown, NCSU, and a researcher, Jianping Wang, U, for their experiments on the CHEETAH network. He managed user accounts, administered the dozen hosts that we have on the CHEETAH network and local testbed (using yum for operating system updates, etc.), and configured VLANs and Layer-1 circuits across the Sycamore SN16000 switches. Several maintenance problems occurred this year. These include a misconfiguration of the OC192 circuit between ORNL and ATL (Oct. 06), disruption of the VLAN circuits 602 and 603 that connected UVa and CUNY to the CHEETAH NC PoP via MPLS tunnels across HOPI and the NCREN network (Oct. 06), failures of the AIC and SMC cards in the ORNL SN16000 (Feb. 07), which required the cards to be sent back to the vendor for replacement after a long period of testing, and a failure of the MCNC-NLR- SLR concatenated Layer-1 circuit between Atlanta and Raleigh (Mar. 07). There were at least three major security-related events, two caused a flood of messages generated by a misconfiguration of network-management software (local incidents), and one caused by a misconfiguration of squid software on the HOPI PCs (which impacted the Internet2 mail server). We summarize our findings based on these activities in Item 7 of the Findings report. 8. CHEETAH wide-area network extension (Tao Li and Xiangfei Zhu): We interconnected the CHEE- TAH network with the HOPI testbed via a 10GbE link as shown in Fig. 2. To enable this link, we relocated the North Carolina (NC) Sycamore SN16000 switch to MCNC s collocation area in Level 3 s Raleigh PoP. This relocation required the purchase of several components, such as a DC-to-AC inverter, a Dell switch to handle a short-reach/long-reach mismatch between our Sycamore SN GbE interface card physical interface and the carrier s requirement, optical fiber, etc. We renewed contracts with SLR for collocation of our Atlanta node and for the intra-city OC192 link within Atlanta between NLR s PoP and SLR s PoP (where our node is located). We renewed our contract with NLR for the Raleigh-Atlanta OC192 link, and added a one-year lease for the Raleigh-Washington 10GbE link. Finally, we entered into a new contractual agreement with MCNC for collocation at their Raleigh 5

6 PoP. Other changes include adding improved fiber GbE NICs to our servers located at the CHEETAH PoPs for more consistent data-plane throughput. Finally the Sycamore switch software was upgraded to Release 8.0 on all three switches. 9. Basic research (Xiangfei Zhu): We first created a DTMC model by approximating the exponential distribution of call inter-arrival times with a geometric distribution. While this model provided us insights into the performance aspects of different types of book-ahead mechanisms (paper accepted to IEEE Transactions of Communications, posted on project web site), its limitation was the state-space explosion problem. With further research, we were able to model the BA-First scheme (in which the first available time interval is allocated to an incoming request) with a non-homogeneous CTMC model, from which we could obtain call congestion, utilization and mean scheduling delay by solving the embedded DTMC model of the system when observed at reservation time-interval boundaries. A paper on this model has been accepted to appear in IEEE Globecom 2007 (posted on the project web site). Our results are presented in Item 9 of the Findings document. 10. Basic research (Xiuduan Fang): In our Globecom 2006 publication, which we described in last-year s annual report, we used the ErlangB call-blocking model to study the immediate-request bandwidthsharing mode for different orders of magnitude of link capacity expressed in channels, i.e., m = 10, 100, 1000 Figure 2: Current CHEETAH network deployment (as of July 21, 2007), and call holding time. This year, we undertook a study using the ErlangC call queueing model for the first type of application, for which the per-circuit capacity is independent of the holding time, and for the second class of applications, i.e., file transfers where the holding time is dependent on per-circuit capacity, we combined well-known M/G/m/m and TCP models to create a new model. Unlike in the Globecom 2006 paper, where call blocking probability and utilization were 6

7 the only metrics of interest, in this paper, we studied mean delays. We submitted this paper to IEEE JSAC, but it was rejected with a mixed set of reviews; we are revising it for a new journal submission (posted on the project web site). Our results are presented in Item 10 of the Findings document. 11. Basic research (Tao Li): We created queueing models of a network of switch call processors and signaling channels to compare in-band and out-of-band signaling options. Our goal is to find a signaling transport mechanism that lowers total call setup delay. Our conclusions are reported in Item 11 of the Findings document. 12. Applications: We tested an open-source CDN (Content Delivery Network) application called Globule CDN application, and a storage backup application called amanda, on our local testbeds. We identified both applications as candidates for dynamically shared circuit-switched networks. We have been meeting regularly with John Lankford, UTK, John Moore, Steve Thorpe, and Mark Johnson, MCNC, to discuss applications for dynamically shared circuit-switched networks (May 07 presentation posted on project web site). MCNC and UTK have an interest in business development for Internet2 s Dynamic Circuit Services (DCS). The amanda application was identified by this group as a potentially attractive application for dynamic circuits. The group is currently working on a business-oriented report based on our discussions. Our conclusions are reported in Item 12 of the Findings document 13. Interceptor software (Helali Bhuiyan): We found a new way to integrate the two components needed to make an application circuit-aware. First, the application needs to invoke the circuit-requestor program to initiate circuit setup. Next, it needs to configure any TCP connection it creates over the circuit to run the CTCP congestion control module, or at least change some of the parameter settings associated with the TCP socket. Our earlier solutions were to either modify the application source code, an approach we adopted with squid, or to write a script-like program that initiates circuit-requestor before starting the application. Our new technique is to use the ptrace (process tracking) tool provided by Linux. An interceptor application can execute the ptrace command with the process identifier (PID) of the traced application. Once a process is being traced, whenever the traced process executes a system call or returns from a system call, the control of execution is transferred to the interceptor application. The interceptor application can then check the arguments of the system call or execute other actions, such as reading registers, modifying register values, or injecting new code into the code segment. Also values returned from the system call can be accessed and modified in a similar way. Once the interceptor application is done examining and/or modifying registers associated with the system call, control is given back to the traced application for continued execution of the system call. With this technique, when an application executes a connect system call to initiate a connection on a TCP socket, the interceptor application can trigger circuit setup, and modify the destination and source IP addresses, if 7

8 required (e.g., when a VLAN is setup the ends of the VLAN virtual circuit are allocated new IP addresses). Control returns to the traced application for execution of the connect system call with the modified parameters. When the system call completes execution, the interceptor program is invoked again, at which point it can configure the socket to run the CTCP congestion control module, or use setsockopt to modify TCP-socket related parameters, such as read and write buffer sizes, before relinquishing control to the traced application. Educational activities include the following. Mark McGinley obtained his Masters degrees this year, and passed the Ph.D. qualifying examination. Xiangfei Zhu defended his Ph.D. proposal successfully. Xiuduan Fang was advanced to candidacy and will be defending her Ph.D. proposal soon. Helali Bhuiyan passed his Ph.D. qualifiers in August Mark and Xiangfei attended the DRAGON/MAX controlplane workshop to learn how to configure GMPLS networks with the DRAGON control-plane software. Undergraduate students, Robert W. Gisiger, Matt Greenfield, Wesley Chi, Thinh Ho, and Marko Milko, did their undergraduate senior-thesis project work on CHEETAH-related projects. Postdoctoral fellow, Tao Li, quickly learned a great deal about maintaining and administering a wide-area experimental network. All the participants learned a great deal about network security. Materials: The materials we developed include papers and software. The publications are listed in the Products part of this report. We published two journal papers, and four conference papers. Software developed in the project along with presentations are being posted on project web site: A list of presentations are provided below. Presentations: Title of Presentation Meeting Place Date CHEETAH applications and control-plane Internet2 J. Techs Meeting Batavia, IL July 16-18, 2007 testing on HOPI (with demonstrations) HOPI Applications MCNC Raleigh, May 8-9, 2007 NC CHEETAH: A high-speed optical network U SEAS Open House Charlottesville, Feb. 24, 2007 Proposals for HOPI testing Generalized MultiProtocol Label Switched (GMPLS) Networks - all-day tutorial Request for feedback on proposed direction for CHEETAH HOPI Meeting, Internet2 J.Techs meeting Globecom, IEEE Communications Society Meeting with NSF Program Directors Minneapolis San Francisco, CA Arlington, Feb. 13, 2007 Dec. 1, 2006 Sept. 22,

9 Title of Presentation Meeting Place Date CHEETAH s use of DRAGON Proposed future direction for CHEETAH CHEETAH project review Proposed future direction for CHEETAH CHEETAH review DRAGON Users Group Presentation Meeting with Tom Lehman, DRAGON, ISI East Review meeting with Program Director Meeting with Jerry Sobieski, DRAGON project, MAX Meeting with Wu Feng and Mark Gardner, V. Tech. Reston, Arlington, Charlottesville, Aug. 30, 2006 Aug. 25, 2006 Aug. 23, 2006 Maryland Aug. 12, 2006 Charlottesville, Aug. 10,