Progress Report. Project title: Resource optimization in hybrid core networks with 100G systems

Transcription

1 Progress Report DOE award number: DE-SC Name of the recipient: University of Virginia Project title: Resource optimization in hybrid core networks with 100G systems Principal investigator: Malathi Veeraraghavan Date of report: June 28, 2012 Period covered by the report: March. 1, May. 31, 2012 Comparison of the actual accomplishments with the goals and objectives established for the period and reasons why the established goals were not met. The three goals established for this period were mostly met. The continued NetFlow analysis was reported in the HNTES II quarterly report. The analysis of the GridFTP logs of NCAR-NICS and SLAC- BNL was carried out in order to test if our selection of 133 Mbps as a threshold to find alpha flows was too high or too low. The ANI testbed experiments were documented and posted at this site: 48eb-b005-fd58ecac54d2 (a user login is required, and will be provided upon request). Also, a tutorial paper is under preparation. A discussion of what was accomplished under these goals during this reporting period, including major activities, significant results, major findings or conclusions, key outcomes or other achievements. This section should not contain any proprietary data or other information not subject to public release. If such information is important to reporting progress, do not include the information, but include a note in the report advising the reader to contact the Principal Investigator or the Project Director for further information. 1 Activities ESnet NetFlow data analysis: New analysis programs were implemented and executed on the anonymized NetFlow reports, raw IP flow and prefix flow data obtained from Chris Tracy for a 7-month period (this data was generated by Chris by executing UVA provided programs on ESnet NetFlow data, and then anonymized). The first program quantified the cost of redirecting /24 and /32 based prefix flows rather than 5-tuple raw IP flows. Data on general purpose flows (beta flows) that share the alpha prefix flow identifiers (and hence would get redirected to the MPLS paths as alpha flows) was extracted. A second program analyzed the percentage of alpha prefix flows that are sourced from or destined to data doors (data transfer servers). A third set of programs characterized alpha flows to quantify per-day prefix flow measures, aggregate prefixflow measures, durations of bursts, raw IP flows and prefix flows, and relative values of alpha flow when compared to all flows. GridFTP usage statistics analysis: As a sanity check on the size and rate thresholds used in the ESnet NetFlow data analysis, we wrote programs to analyze GridFTP usage statistics. These statistics provide per-transfer size and duration information from which a transfer rate can be estimated. We also ran experiments on the ANI testbed to understand how GridFTP client 1

2 arguments, such as fast and p, influence the reuse of TCP connections for multiple file transfers in a session. ANI testbed: The five traffic engineering experiments we ran on the ANI testbed were documented: (i) connectionless packet forwarding with IP routing, (ii) connectionless packet forwarding with integrated Ethernet switching and IP routing, (iii) connection-oriented end-toend VLAN based virtual circuits, (iv) connection-oriented end-to-end VLAN-MPLS-VLAN hybrid virtual circuits, and (v) hybrid connectionless/connection-oriented paths with Layer 3 IP packet forwarding on to MPLS virtual circuits. Additional documentation was created for two QoS experiments: (i) Diffserv solution, and (ii) MPLS solution. New iperf experiments were conducted to study the influence of policer setting on packet losses. Simulation study: We resumed our simulation study to characterize the adverse effects of alpha flows on real-time delay/jitter sensitive audio/video flows. Our starting point was a literature search to determine the QoS requirements and traffic characteristics of audio/video flows. 2 Findings ESnet NetFlow data analysis: The beta flow data analysis showed that because most high-end data transfer nodes are set up as clusters in their own subnets, most beta flows are from data transfer applications, and not from general-purpose applications such as Web browsing and . To make this determination, beta NetFlow reports were filtered out in sequence into these four categories: (i) reports with 5-tuples that matched those of alpha flows (recall that a flow is defined as an alpha flow if in any of its NetFlow reports its size exceeds 1 GB, but such a flow could have other NetFlow reports with sizes below this threshold), (ii) reports with more than 1 packet that are of size greater than 1000 bytes, and have at least one other report with the same 4-tuple, which includes source and destination IP addresses, protocol type, and one of the two port numbers (these flows are deemed as file transfer flows ), (iii) reports for flows with well-known port numbers, specifically SSH, HTTP, IMAP, SMTP, SSMTP, HTTP, HTTPS, NNTP, IMAP, IMAPS, IMAP4SSL, Unidata, RTSP, rsync, SFTP, BFTP, FTPS, POP3, and SSLPOP (some file transfers use SCP, which would appear as an SSH flow, but such flows would have been filtered out into the first or second categories leaving behind only those SSH flows corresponding to interactive applications such as SecureCRT), and (iv) remaining (unclassifiable) reports. For the 214-day period for which data was analyzed, 12.31% (for /24) or 33.01% (for /32) of beta flows packets fell in the first category, i.e., these flows shared the same 5-tuple as an alpha flow. Most of the beta flow packets belonged to the second category: for the /24 case, 79.76% of the beta flows packets were classified as file transfer flows, and for the /32 case, the percentage was Only a small percentage of beta flows packets belonged to the third category, i.e., from non-file transfer flows with well-known port numbers: for the /24 case, this number was %, while for the /32 case, the number was 0.014%. The fourth category (unknown) percentage was small (e.g., 8% for the /24 case). GridFTP usage statistics analysis findings: The SLAC GridFTP log data analysis showed that 15.4% transfers are 1-stream transfers (use one TCP connection) and 84.6% transfers are 8- stream transfers (use 8 parallel TCP connections). Upon grouping the transfers into these two categories and then computing their rates, we found that 21.7% of the 1-stream transfers have 2

3 throughput higher than 133 Mbps (alpha flow threshold). But only 0.48% of the 8-stream transfers exceeded Gbps (133 Mbps 8). This suggests that the 1 GB in 1 min threshold is perhaps too high for the type of transfers executed with GridFTP. Admittedly, this is only one data point (SLAC), but it demonstrates a need for sensitivity analysis for this threshold value. ANI testbed: In the new iperf experiments conducted to study the effect of rate-policing configuration on packet losses, we found that with a policy was to discard packets exceeding a certain rate, there were more packet losses than with a policy to mark rate-exceeding packets with the DSCP code for scavenger service (which was configured on the output interface to be served at a 5% rate). The interface counter statistics were obtained to make this determination. This finding will influence the design of steps 2 and 3 of HNTES, i.e., LSP provisioning and firewall filter configuration. Simulation study: A literature search to determine the most commonly accepted A/V requirements was conducted. For one-way delay, 0-25ms is required for operation without echo cancellers, ms is required for high-quality calls in the presence of echo cancellers, and ms is acceptable for satellite-link calls. Delays greater than 400ms should be avoided. Loss requirements vary from 1% to 5%. Losses occur when packets are delayed beyond the expected playout time. Adaptive playout buffers are used for packet voice. The goal of the simulation is to characterize, preferrably analytically, the minimum size and rate thresholds at which alpha flows start causing A/V flows to miss their requirement targets. These thresholds will depend on link rates, buffer size, and background traffic. The arrival pattern of alpha flows within the duration of an A/V flow is also important because every time a burst of alpha flow packets occupy a router buffer, an A/V packet will be delayed causing it to miss its deadline for playout at the receiver. Cost Status. Show approved budget by budget period and actual costs incurred. If cost sharing is required break out by DOE share, recipient share, and total costs. Please see attached file. Schedule Status. List milestones, anticipated completion dates and actual completion dates. If you submitted a project management plan with your application, you must use this plan to report schedule and budget variance. You may use your own project management system to provide this information. As per the document, dated Oct. 14, 2009, which was sent to the program manager, Dr. Thomas Ndousse-Fetter, and is available on the project web site, the milestone for this quarter is as follows: Aug. 31, 2012 Software demonstrations and reports of HNTES Next quarter deliverables: By the end of the next quarter, Aug. 31, 2012, the following deliverables will be completed: NetFlow analysis will be completed for the 7-month data and a journal paper submitted. 3

4 GridFTP analysis: Correlation between NetFlow data and GridFTP data will be undertaken. ANI testbed experiments will be undertaken to characterize packet losses under different policing configurations, transmission rates and buffer size allocations in the virtual queues for diffferent QoS classes. An ns2 simulation will be set up with background traffic, arriving and departing alpha flows, and a single long-held audio/video flow. Simulations will be run, and data collected. Any changes in approach or aims and reasons for change. Remember significant changes to the objectives and scope require prior approval by the contracting officer. Actual or anticipated problems or delays and actions taken or planned to resolve them. Any absence or changes of key personnel or changes in consortium/teaming arrangement. None A description of any product produced or technology transfer activities accomplished during this reporting period, such as: A. Publications (list journal name, volume, issue); conference papers; or other public releases of results. Z. Yan, C. Tracy, M. Veeraraghavan, "A Hybrid Network Traffic Engineering System," accepted in IEEE HPSR 2012, June 24-27, J. Li, M. Veeraraghavan, A Reliable Message Multicast Transport Protocol for Virtual Circuits, th International Conference on Communications, Mobility, and Computing (CMC 2012), May 2012, pp Jie Li, Matthew Manley, Malathi Veeraraghavan, Steve Emmerson, Analysis and selection of a network service for a scientific data distribution project, th International Conference on Communications, Mobility, and Computing (CMC 2012), May 2012, pp Z. Yan, Z. Liu, C. Tracy, and M. Veeraraghavan, Hybrid network traffic engineering system (HNTES), ESCC Meeting at Joint Techs Baton Rouge, LA Jan , Z. Yan, Z. Liu, C. Tracy, and M. Veeraraghavan, Hybrid Network Traffic Engineering System, Talk presented at Annual DOE PI meeting for the ASCR Network & Middleware, Mar. 1-2, Z. Liu, M. Veeraraghavan, Z. Yan, C. Tracy, J. Tie, I. Foster, J. Dennis, J. Hick, Y. Li and W. Yang, On using virtual circuits for GridFTP transfers, submitted to SC

5 B. Web site or other Internet sites that reflect the results of this project. UVA Hybrid Networking Project web site: Collaboration web site for project participants to post documents, discuss issues in an online forum, archive s, etc. The password-protected site is: C. Networks or collaborations fostered. We are working closely with the Chris Tracy, ESnet, and the ANI Tabletop Testbed design team, including Brian Tierney, Inder Monga, Chin Guok and Eric Pouyoul, ESNet. D. Technologies/Techniques. HNTES2.0 software modules, such as OFATv3, have been developed. They are currently under test. As noted earlier, the software will be made available via our open project web site. E. Inventions/Patent Applications F. Other products, such as data or databases, physical collections, audio or video, software or netware, models, educational aid or curricula, instruments or equipment. HNTES software, and results of traffic analysis, are available through our project web site. 5