America Connects to Europe (ACE) (Award # ) Year 7 Annual Report 1- Mar through 31- May Jennifer Schopf Principal Investigator

Transcription

1 America Connects to Europe (ACE) (Award # ) Year 7 Annual Report 1- Mar through 31- May Jennifer Schopf Principal Investigator Summary During Year 7 of the America Connects to Europe (ACE) project, March 2016 through May 2017, the project concluded its collaborative and engineering activities to support international research. This report outlines collaborations, software and systems work, operational activities, and usage statistics for the project. Over the course of the final year, many development and outreach efforts were scaled back in order to ensure support for the circuits until a replacement could be put in place. However, we continued to support science engagement and measurement and monitoring throughout. 1. ACE Overview The America Connects to Europe (ACE) project supported a series of circuits and services between the US East coast and Europe. In the final year of the project, these circuits consisted of: Three 10G circuits between WIX (McLean, VA) and Frankfurt. These circuits were lagged together and load balanced for performance, and sometimes were reported on as a 30G single unit. All three of these were funded by NSF. Traffic began shifting off of these links in December 2016, and the circuits were shut down in February GÉANT s Paris to MAN LAN 100G circuit. GÉANT has completed the incorporation of their circuit into the Advanced North Atlantic consortium. These circuits were used in production to support a wide variety of science applications in addition to supporting all traffic from Africa to the US. In addition, through a supplement to the award, the ACE project directly supported application use of 100G test beds and tool development for 100G links. Overall, this award supported tool development, software defined networking (SDN) experimental work, and measurement and security activities. 1

2 2. Staffing At the start of the year, project staff consisted of: Jennifer Schopf, Director Andrew Lee, primary ACE senior network engineer Scott Chevalier, perfsonar Everywhere main support Matt Robinson, primary contact for GlobalNOC support desk During the year, this effort was cut back over time to ensure support for the circuits. 3. Collaborations, Travel, and Training ACE staff continued to collaborate with partners for better understanding of collaborative science use of the ACE links and supporting use of emerging network technologies. Almost of these trips were funded by other projects, but were relevant to the ACE mission. As detailed in previous reports, these collaborations included: Lee attended the Large Hadron Collider Open Networking Environment (LHCONE) and the International Symposium of Grids and Clouds (ISGC) meeting March 13-18, Taipei, Taiwan ( Radulovic attended CENIC 2016 annual meeting in Davis, CA, March 21-23, ( Schopf and Radulovic attended Cross- Connect: Bioinformatics workshop held April 11-12, 2016, in Berkeley, CA ( engagement/programs- and- workshops/crossconnects- workshop- series/). Chevalier, Lee, and Addleman participated in a perfsonar workshop on May 10th and 11th. Schopf, Lee, and Addleman attended the Internet2 Global Summit ( global- summit/) in Chicago during the week of May 15th. Lee and Schopf attended the TNC16 conference in Prague on June Lee, Addleman, and Chevalier attended an Operating Innovative Networks or OIN training in Indiana on July Radulovic participated in a Galaxy Conference held at IU- Bloomington in July Schopf attended CASC in Washington, DC September Radulovic attended LHCONE meeting held in Helsinki, Finland on Sep 20 and 21, collocated with the Nordunet conference. Schopf and Addleman attended the Internet2 Technology Exchange Meeting in Miami September Lee attended the GLIF meeting September 27-30, 2016 in Miami. Addleman attended the CACR Cyber Security Summit in Indianapolis on September 29. Schopf attended the UbuntunetConnect meeting in Kampala, Uganda. Schopf, Lee, Radulovic, and Chevalier attended SuperComputing 2016 in Salt Lake City November 13-18,

3 Addleman attended a SANS Security Course November 27-30, 2016 Schopf and Radulovic attended the Winter ESIP meeting in Bethesda, Maryland on January Schopf met with Christopher Whalen at NIH to discuss future collaborations with NIH- supported clinical work in Africa. Schopf and Lee attended the Second Annual Hawaii Meeting on Trans- Pacific R&E Networking held to discuss Guam and other Pacific networking issues. Schopf and Lee attended the Pacific Telecommunications Council meeting in Honolulu on January 15-18, Schopf and Radulovic, along with members of the ESNet Science Engagement Team, spent a day with Camille Crittenden at UC Berkeley to discuss future collaborations on science engagement. Schopf and Chevalier attended the January perfsonar planning meeting held at LBNL on January Schopf attended the Quilt Regional Networking winter meeting in San Diego on February event/2017- winter- member- meeting- february la- jolla- ca/. Radulovic attended the PRP winter meeting on science engagement at UCSD on February Lee and Addleman ran a multi- day workshop similar to the OIN workshops at the PregiNET office, Manila, Philippines, March 6-9. Addleman was an instructor for the Operating Innovative Networks (OIN) training in Syracuse, March 14-15, Schopf and Radulovic attended the CENIC meeting in San Diego on March Schopf attended the WACREN annual meeting in Abidjan, Ivory Coast, on March Lee attended GlobusWorld in Chicago on April Schopf attended the Internet2 Global Summit on Washington, DC, on April This included a meeting of the IRNC PIs which Lee also attended. Schopf, Moynihan, and Lee attended TNC in Austria on May 29- June Software and Systems Work A. Tool Development The development of GlobalNOC tools, funded in small part by ACE, continued this year with patches and upgrade to existing tools. During the last year of the project, all other development work was curtailed in order to ensure the continuation of the circuits for the final year. B. Software Defined Networking (SDN) Activities The ACE network supported SDN and OpenFlow as part of our partnership with Internet2 and GÉANT. There was an Inter Domain Controller (IDC) at WIX, which made it possible to create dynamic circuits on ACE related circuits through those 3

4 exchanges using OSCARS and enabled us to provide Layer 2 Virtual Circuits dynamically instead of relying on static services. US researchers using the Internet2 AL2S network were able to create dynamic circuits to Europe using the ACE networks. GÉANT offers a similar service to AL2S called AutoBAHN that allows users to dynamically provision services across the GÉANT network. C. PerfSONAR Everywhere and Small Nodes Small perfsonar nodes to support smaller, cheaper, and more portable test points. Funding for the perfsonar small node project ended in December after a successful deployment on 8 sites with ilight. This project was also discussed at several of the meetings attended. 5. Operational Activities A. Circuit Procurement/Decommission The 30G LAG was discontinued in February, 2017, although we began shifting traffic away from this link in December, The bulk of the traffic from this link is now flowing over the 100G NEAAR circuit that runs between ManLan and the London Open Exchange. B. Graphs Figures 1 and 2 show the aggregated traffic for the three 10G circuits from WIX to Frankfurt. The circuits are lagged together to create a total capacity of 30Gbps. Figures 4 and 5 show the traffic for the GÉANT 100G circuit between ManLan and Paris. Figure 1: Aggregated traffic using smoothed daily values on the 30G Lag between WIX and Frankfurt for Year 7, March 1, 2016 through May 31,

5 Figure 2: Aggregated traffic using maximum average daily values on the 30G Lag between WIX and Frankfurt for Year 7, March 1, 2016 through May 31,

6 Figure 3: Aggregated traffic using smoothed average daily values on the GÉANT 100G circuit between ManLan and Paris for Year 7, March 2017 to May

7 Figure 4: Aggregated traffic using maximum daily values on the GÉANT 100G circuit between ManLan and Paris for Year 7, March 2016 to May

8 Table 1 shows full volume of traffic transferred over the ACE links during the 5th quarter. Over 8 Petabytes of traffic has been transferred over the ACE links during the 3 months. Table 2 shows full volume of traffic transferred over the ACE links during the final year of the project. Over 61 Petabytes of traffic has been transferred over the ACE links during the project year. Table 1: Traffic in terabytes transferred over ACE links, March 1, 2017 through May 31, Mar Apr May Total Frankfurt- >WIX WIX- >Frankfurt Paris- >MAN LAN MAN LAN- >Paris Total: Table 2: Traffic in terabytes transferred over ACE links for Year 7, March 1, 2016 through May 31, Q1 Q2 Q3 Q4 Q5 Total Frankfurt- >WIX WIX- >Frankfurt Paris- >MAN LAN MAN LAN- >Paris Total: In December 2017, the equipment collecting flow data for the ACE circuits was removed from WIX in order to be redeployed as part of the NEAAR circuit. Therefore, we do not have flow data statistics to report for Quarter 5. However, Figures 5 through 8 show the top ten talker data for Year 7 of the project combined. 8

9 Figure 5: Top Talker data for the 30G ACE LAG for Year 7, Inbound by source. 9

10 Figure 5: Top Talker data for the 30G ACE LAG for Year 7, Inbound by destination. Figure 6: Top Talker data for the 30G ACE LAG for Year 7, Outbound by source. 10

11 Figure 7: Top Talker data for the 30G ACE LAG for Year 7, Outbound by destination. C. Trouble Tickets During Year 7, there were 23 tickets for scheduled maintenance, detailed in Table 3, and 60 tickets for unscheduled maintenance, detailed in Table 4. Additional trouble ticket information is available at Table 3: Tickets for scheduled maintenance on ACE circuits, March 1, 2016 through May 31, 2017a. Ticket Number Customer Impact Network Impact Title Type Source Of Impact Start Time (UTC) End Time (UTC) Elevated 3- Elevated Emergency Completed- ACE GLOBAL- FRK- Circuit Vendor 05/09/2016 6:01 AM 05/09/2016 6:02 AM 764 GLOBAL- FRK- Circuit Vendor 06/08/2016 2:01 AM 06/08/2016 5:08 AM 765 GLOBAL- FRK- Circuit Vendor 06/10/2016 2:11 AM 06/10/2016 3:57 AM 771 Circuit Vendor 06/11/2016 6:06 AM 06/11/2016 3:57 PM 11

12 GLOBAL- FRK- GLOBAL- FRK- GLOBAL- FRK- GLOBAL- FRK- GLOBAL- FRK- GLOBAL- FRK- Emergency 1 of 2 GLOBAL- FRK- 1 of 2 GLOBAL- FRK- 3- Elevated 1- Critical Emergency Core Node (WASH) 3- Elevated 1- Critical Emergency Core Node (WASH) GLOBAL- FRK- Circuit Vendor 06/15/2016 2:16 AM Circuit Vendor 07/01/2016 2:06 AM 07/01/2016 5:22 AM Circuit Vendor 07/06/2016 2:04 AM Circuit Vendor 07/08/2016 3:09 AM Circuit Vendor 07/15/2016 3:06 AM Circuit Vendor 07/30/2016 5:09 AM Circuit Vendor 07/30/2016 3:33 PM Hardware Internal 08/17/2016 1:00 AM Hardware Internal 08/23/2016 4:10 AM Circuit Vendor 09/15/2016 8:00 PM 06/15/2016 5:55 AM 07/01/2016 5:01 AM 07/01/2016 5:23 AM 07/06/2016 6:00 AM 07/08/2016 7:39 AM 07/15/2016 7:00 AM 07/30/2016 6:00 AM 07/30/2016 6:37 PM 08/17/2016 4:14 AM 08/23/2016 4:13 AM 09/15/2016 8:08 PM 844 Circuit Vendor 10/28/ /28/

13 GLOBAL- FRK- 4:16 AM 5:44 AM 849 Emergency GLOBAL- FRK- Circuit Vendor 11/11/ :04 AM 11/11/ :11 AM 853 GLOBAL- FRK- Circuit Vendor 12/01/2016 4:01 AM 12/01/2016 6:53 AM 857 GLOBAL- FRK- Circuit Vendor 12/07/ :01 PM 12/07/ :34 PM 864 Emergency GLOBAL- FRK- Circuit Vendor 12/13/ :06 AM 12/13/ :18 AM 871 GLOBAL- FRK- Circuit Vendor 01/14/2017 8:52 AM 01/14/2017 9:39 PM 898 GLOBAL- FRK- Hardware Vendor 01/29/2017 4:04 AM 01/29/2017 8:06 AM 903 GLOBAL- FRK- Circuit Vendor 02/07/2017 4:03 AM 02/07/2017 4:17 AM 920 Emergency GLOBAL- FRK- Circuit Vendor 02/19/2017 4:36 AM 02/19/2017 4:41 AM 13

14 Table 4: Unscheduled outage tickets for the ACE circuits, March 1, 2016 through May 31, Ticket Number Customer Impact Network Impact Title Outage Type Source Of Impact Start Time (UTC) End Time (UTC) Normal 1- Critical Outage Resolved - ACE Various Circuits 4- Normal 2- High Outage Resolved- ACE 4- Normal 2- High Outage Resolved - ACE s & WIX High 2- High Instability - ACE 2- High 2- High Instability - ACE - ACE - ACE Hardware Internal 03/04/ :41 PM Vendor 04/20/2016 6:16 AM Vendor 05/09/2016 1:28 PM Vendor 06/01/2016 6:25 AM Power Vendor 06/03/ :01 PM Vendor 06/11/2016 4:01 PM Vendor 06/13/2016 4:53 AM Vendor 06/17/2016 2:07 AM Vendor 06/29/2016 2:03 AM Undetermined Vendor 06/30/ :16 AM 03/04/2016 9:15 PM 04/20/2016 1:28 PM 05/10/2016 2:29 AM 06/02/2016 5:38 AM 06/03/2016 1:35 PM 06/11/2016 4:03 PM 06/14/2016 6:26 AM 06/17/2016 4:11 AM 06/29/2016 4:15 AM 06/30/ :12 PM 14

15 WIX- 10GE ACE 2- High 2- High Outage Resolved- ACE - ACE - ACE - ACE 2- High 2- High Outage Resolved- ACE 2- High 2- High Availability - ACE - ACE 2- High 2- High Instability - ACE Vendor 07/01/2016 2:06 AM Hardware Vendor 07/02/2016 1:25 PM Vendor 07/06/2016 6:00 AM Vendor 07/15/2016 3:06 AM Power Vendor 07/15/ :31 PM Hardware Vendor 07/17/2016 4:03 PM Power Vendor 07/19/2016 4:11 AM Vendor 07/27/2016 9:00 PM Vendor 07/31/ :44 PM Vendor 08/05/2016 9:07 PM 07/01/2016 5:23 AM 07/02/ :12 PM 07/06/2016 7:33 AM 07/15/2016 7:39 AM 07/15/2016 3:17 PM 07/17/2016 5:07 PM 07/19/2016 5:31 AM 07/27/2016 9:27 PM 08/08/2016 4:06 AM 08/08/2016 4:06 AM 15

16 High 2- High Availability - ACE - ACE - ACE Various LAG s - ACE - ACE 2- High 2- High Stability - ACE - ACE - ACE - ACE - ACE Undetermined Vendor 08/17/2016 9:48 AM Bumped Fiber Internal 08/17/2016 1:26 PM Vendor 09/01/2016 4:00 AM Vendor 09/06/2016 2:35 PM Vendor 09/07/2016 4:43 PM Undetermined Vendor 09/09/2016 1:34 PM Hardware Vendor 09/11/2016 3:31 AM 09/12/2016 5:50 PM 09/13/ :41 PM Vendor 09/15/2016 7:54 PM Undetermined Vendor 09/19/ :04 AM Vendor 09/26/2016 6:39 AM Power Vendor 10/08/2016 6:04 PM 08/17/ :12 AM 08/17/2016 1:36 PM 09/01/2016 6:32 AM 09/06/2016 2:42 PM 09/08/2016 4:35 AM 09/09/2016 4:04 PM 09/11/2016 3:32 AM 09/12/2016 5:51 PM 09/13/ :42 PM 09/15/2016 8:00 PM 09/19/ :05 AM 09/30/2016 9:04 PM 10/08/2016 8:32 PM 16

17 & 842 Bumped Fiber Vendor 10/10/2016 4:54 PM 10/10/2016 4:55 PM Normal 1- Critical Brief Outage Various s Vendor 10/13/2016 5:29 PM 10/13/2016 5:36 PM 845 Resolved- ACE Power Vendor 10/25/2016 3:41 PM 10/25/2016 3:45 PM ACE and Vendor 11/01/2016 7:39 PM 11/02/2016 3:17 AM High 2- High Stability - ACE Vendor 11/02/2016 7:04 AM 11/02/2016 7:05 AM 11/02/ :03 AM 11/02/ :04 AM 11/02/ :12 AM 11/02/ :13 AM 851 Bumped Fiber Vendor 11/11/2016 5:33 AM 11/11/2016 5:34 AM 11/11/2016 6:36 AM 11/11/2016 6:37 AM 850 Bumped Fiber Vendor 11/11/2016 5:39 AM 11/11/2016 5:40 AM 11/11/2016 9:31 AM 11/11/2016 9:32 AM 17

18 ACE - ACE Various s - ACE 2- High 2- High Instability - ACE - ACE Various Circuits 2- High 2- High Stability - ACE 2- High 2- High Stability - ACE Customer GEANT s ACE- WIX, FRK- WASH2, Vendor 11/14/ :00 PM Undetermined Vendor 11/22/2016 7:04 AM Undetermined Undetermined 12/05/ :20 AM Hardware Vendor 12/13/2016 2:27 PM 12/13/2016 9:23 PM 12/13/ :49 PM Vendor 12/31/2016 3:56 AM Power Vendor 01/03/2017 9:00 AM Power Internal 01/03/2017 1:41 PM Power Vendor 01/08/2017 1:38 PM Undetermined Undetermined 01/08/2017 1:38 PM 01/09/2017 5:27 PM 01/10/2017 2:04 AM Hardware Vendor 01/16/ :44 PM 11/14/ :17 PM 11/22/2016 7:06 AM 12/05/ :21 AM 12/13/2016 4:27 PM 12/13/2016 9:34 PM 12/13/ :18 PM 12/31/2016 6:21 AM 01/03/2017 9:01 AM 01/03/2017 3:26 PM 01/10/2017 3:39 PM 01/08/2017 1:39 PM 01/09/2017 5:28 PM 01/11/2017 3:50 AM 01/16/ :45 PM 18

19 FRK- WASH ACE Power Vendor 01/24/2017 9:31 AM 01/24/2017 9:32 AM 01/24/2017 9:35 AM 01/24/2017 9:36 AM 01/24/2017 9:37 AM 01/24/2017 9:38 AM 901 Undetermined Vendor 01/25/2017 1:47 PM 01/25/2017 1:48 PM 902 Bumped Fiber Vendor 01/27/2017 4:57 PM 01/27/2017 4:58 PM ACE Vendor 02/01/2017 4:16 AM 02/01/2017 4:17 AM ACE Various Circuits Vendor 02/01/ :50 PM 02/02/2017 4:10 AM ACE Undetermined Vendor 02/03/2017 2:01 AM 02/03/2017 2:02 AM 907 Vendor 02/06/2017 4:35 PM 02/06/2017 4:36 PM ACE Various s Vendor 02/14/2017 1:45 PM 02/14/2017 9:51 PM 02/15/ :26 AM 02/17/2017 1:39 AM 928 Power Vendor 02/20/2017 2:18 PM 02/20/2017 2:19 PM ACE Various Power Vendor 02/23/ :04 AM 02/23/ :06 AM 19

20 Circuits - ACE - ACE and Global- ACE- WIX- OC Power Vendor 02/27/2017 9:02 PM 02/28/2017 5:36 PM Power Vendor 02/28/2017 2:50 AM 02/28/2017 5:36 PM 02/27/2017 9:03 PM 02/28/2017 5:42 PM 02/28/2017 2:51 AM 02/28/2017 5:39 PM D. Downtime and Availability Table 5: Downtime and availability for ACE core nodes, March 1, 2016 through February 28, ACE Core Nodes Down Time Reporting Period Availability 52 Week Availability sw.wash.ace.iu.edu 3 hr 16 min 99.96% 99.96% Aggregate ACE Core Nodes 3 hr 16 min 99.96% 99.96% Table 6: Downtime and availability for ACE backbone circuits, March 1, 2016 through February 28, ACE Circuits Down Time Reporting Period Availability 52 Week Availability ACE-FRK-WASH-10GE hr 54 min 97.67% 97.69% ACE-FRK-WASH-10GE hr 57 min 97.59% 97.61% ACE-FRK-WASH-10GE hr 47 min 97.75% 97.77% GLOBAL-ACE-WIX-10GE hr 48 min 99.71% 99.71% GLOBAL-ACE-WIX-10GE hr 48 min 99.71% 99.71% GLOBAL-ACE-WIX-10GE hr 44 min 99.69% 99.70% GLOBAL-ACE-WIX-O hr 29 min 97.27% 97.28% GLOBAL-FRK-WASH2-10GE- 227 hr 57 min 97.40% 97.41% GLOBAL-FRK-WASH2-10GE- 245 hr 7 min 97.20% 97.22% Aggregate All ACE Circuits 1402 hr 31 min 98.22% 98.23% 20

21 6. Security Events and Activities Basic security measures were maintained during the final year and no security incidences were reported. Basic security work is included as part of the ongoing funded support ACE receives from the GlobalNOC. 21