Hong Kong Workshop on Next Generation Open Communication Exchanges for Science Research University of Hong Kong. January 18, 2012

Transcription

1 Advanced Communication Exchange Facilities: Architecture, Services, and Design, Technologies, Implementation and Operations (i.e., Creating 21 st Century Communications Services and Technology) Joe Mambretti, Director, International Center for Advanced Internet Research ( Northwestern University Director, Metropolitan Research and Education Network ( Hong Kong Workshop on Next Generation Open Communication Exchanges for Science Research University of Hong Kong January 18, 2012

2 Creating the Future of Communications Close Examination of Emerging and Future Requirements New Architecture New Services New Technology Testing With Simulations and Models Testing via Network Research Testbeds At Scale! Best Window To the Future = Data Intensive Science Research 30+ Year History of Communication Innovations Has Been Driven Primarily By Data Intensive Sciences These Sciences Encounter Technology Issues Many Years Before Other Domains

3 NG Digital Sky Survey ATLAS ALMA: Atacama Large Millimeter Array DØ (DZero) www-d0.fnal.gov IVOA: International Virtual Observatory ANDRILL: Antarctic Geological Drilling GEON: Geosciences Network LIGO OSG BIRN: Biomedical Informatics Research Network GLEON: Global Lake Ecological Observatory Network WLCG lcg.web.cern.ch/lcg/publi c/ Globus Alliance CAMERA metagenomics camera.calit2.net Carbon Tracker gmd/ccgg/carbontrack er OOI-CI ci.oceanobservatories.org Pacific Rim Applications and Grid Middleware Assembly SKA rg Sloan Digital Sky Survey Compilation By Maxine Brown CineGrid ISS: International Space Station n TeraGrid XSEDE LHCONE Comprehensive Large-Array Stewardship System

4 A Brief Overview of the Future of Communications Transition From Monolithic, Static Services and Infrastructure To Ad Hoc Specialized, Customized, and Even Personalized Resources: E.g., Personal Global Networks Individualized Communication Services Note: Historic Progression From Monolithic To Individualized Environments (Part of the IT DNA) Personal Computer vs Mainframe Smart Phone vs Personal Computer Intelligent Devices vs Smart Phones Etc. Advanced Network Testbeds Are Being Designed And Implemented To Accomplish These Goals, Including By The National Science Foundation

5 Paradigm Shift Ubiquitous Services Based on Large Scale Distributed Facility vs Isolated Services Based on Separate Component Resources Traditional Provider Services: Invisible, Static Resources, Centralized Management, Highly Layered Distributed Programmable Resources, Dynamic Services, Visible & Accessible Resources, Integrated As Required, Non-Layered Invisible Nodes, Elements, Hierarchical, Centrally Controlled, Fairly Static Unlimited Services, Functionality, Limited Services, Functionality, Flexibility, Expandability Flexibility, Expandability Releasing the Fully Potential of Digital Technologies

6 Next Phase = A Global Platform For Customized Networks, Led By Science Networks Migration from Designing, Creating, Implementing, Operating Private Networks As Individual Projects To Designing, Creating, Implementing, Operating Distributed Environment (Platform) Within Which Such Individual Networks Can Be Readily Created Existing Building Blocks. Prototypes, Including the StarLight National/International Communications Exchange Facility, Metropolitan Research and Education Network (MREN, Regional), National Lambda Rail (NLR, National), Global Lambda Integrated Facility (GLIF, International), GLORIAD (Global Ring for International Application Development, International)

7 Distributed Programmable Network Platforms Programmable Network Platforms = Capabilities for Creating Instant New and Enhanced Services vs Legacy Multi-Year Schedule of Design, Development, and Deployment Advanced = Dynamic vs Static Highly Customizable, Including At Edge High Level of Abstractions, Including APIs Flexible Middleware Processes That Can Be Dynamically Provisioned Highly Distributed Processes vs Centralized Command and Control Etc Programmable = All Resource Elements As Objects Discoverable/Integratable Programmability Extending To Hardware Components Rich Semantics for Resource Discovery and Integration

8 Ad Hoc Specialized Networks APN Approach: Create Private Network Based On Controllable Resources, e.g., Private Optical Fiber/Lambdas/L2 VLANs Advanced Control Planes/Advanced Control Frameworks Advanced Management Planes Possible To Obtain Leverage From IaaS/NaaS PaaS SaaS OaaS XaaS Additional Leverage from Dynamic Clouds Closely Integrated With Dynamic Networks (Ref TransCloud, Note Demo At GEC10)

9 Control Framework Architecture and Implementations Control Frameworks Enable Resources Programming Many Approaches To Control Framework Architecture Currently Being Created By Network Research Communities Many More Will Be Created By Research Communities One Approach Currently Being Transitioned From Research Communities To A Standard Through a Joint Project Between the GLIF and the Open Grid Forum (a Standards Organization) Network Services Interface (NSI) Based In Part On GLIF GOLE Control Frameworks and an API (FENIUS)

10 Fenius GLIF Demonstrations Global Lambda Grid Workshop SC10 HK GLIF Technical Workshop t

11 Automated GLIF Open Lambda Exchange Demonstration at SC11

12 Other Network Control Frameworks Many Other Network Control Frameworks Are Being Created (About 24 Major Initiatives) The US National Science Foundation Has Established an Initiative That Incorporates 4 Additional Major Frameworks The Initiative Is The Global Environment for network Innovations (GENI)

13 National Science Foundation s Global Environment for Network Innovations (GENI) GENI Is Funded By The National Science Foundation s Directorate for Computer and Information Science and Engineering (CISE) GENI Is a Virtual Laboratory For Exploring Future Internets At Scale. GENI Is Similar To Instruments Used By Other Science Disciplines, e.g., Astronomers Telescopes, HEP - Synchrotrons GENI Creates Major Opportunities To Understand, Innovate and Transform Global Networks and Their Interactions with Society. GENI Is Dynamic and Adaptive. GENI Opens Up New Areas of Research at the Frontiers of Network Science and Engineering, and Increases the Opportunity for Significant Socio-Economic Impact. GENI Is Both A Research Instrument and A Prototype of Future Programmable Network Platforms

14 igeni: The International GENI The igeni Initiative Is Designing, Developing, Implementing, and Operate a Major New National and International Distributed Infrastructure. igeni Is Placing the G in GENI Making GENI Truly Global. igeni Is a Unique Distributed Infrastructure Supporting Research and Development for Next-Generation Network Communication Services and Technologies. This Infrastructure Has Been Integrated With Current and Planned GENI Resources, and Is Being Operated for Use by GENI Researchers Conducting Experiments That Involve Multiple Aggregates At Multiple Sites. igeni Infrastructure Connects Its Resources With Current GENI National Backbone Transport Resources, With Current and Planned GENI Regional Transport Resources, and With International Research Networks and Projects.

15 igeni/gcdnet National/International Fabric

16 SC11 SCinet Research Sandbox OpenFlow Demonstrations Seattle, Wash November 2011

17 TransCloud Experiments Alvin AuYoung, Andy Bavier, Jessica Blaine, Jim Chen, Yvonne Coady, Paul Muller, Joe Mambretti,Chris Matthews, Rick McGeer, Chris Pearson, Alex Snoeren, Fei Yeh, Marco Yuen Example of working in the TransCloud [1] Build trans-continental applications spanning clouds: Distributed query application based on Hadoop/Pig Store archived Network trace data using HDFS Query data using Pig over Hadoop clusters [2] Perform distributed query on TransCloud, which currently spans the following sites: HP OpenCirrus Northwestern OpenCloud UC San Diego Kaiserslautern Use By Outside Researchers? Yes Use Involving Multiple Aggregates? Yes Use for Research Experiments? Yes Also Ref: Experiments in High Perf Transport at GEC 7 Demo:

18 Digital Media TransCoding Demonstration Transcoding Cloud 1 TransCloud: Advanced Distributed Global Environment Enables Dynamic Creation of Communication Services, Including Those Based on Rapid Migration of Virtual Network and Cloud Resources TransCloud: Set of Protocols, Standards, Management Software Enables Interoperation of Distinct Cloud and Network Resources Example: Dynamic Cloud+Dynamic Network for Digital Media Transcoding Using Single Platform vs Multiple Infrastructures Transcoding Cloud 2 Video Sources Transcoding Cloud 3 OpenFlow Switches

19

20 JGN-X Network Topology National Institute for Information Communication Technology (NICT) National International Testbed 10 Gbps Connection via GLIF/TransLight StarLight

21 CNIC/Chinese Academy of Sciences icair Signing Ceremony

22 Another Driver For Programmable Network Platforms With Advanced Control Frameworks Advanced Digital Media Many Services Have Requirements Similar To Large Scale Science Visualization Is a Key Application. Currently, Many Major Innovations In Advanced Digital Media

23

24 Programmability + High Capacity Advanced Programmability Provides A Wide Range of Capabilities As Basic Resources, Network Resource Objects Can Be Encapsulated and Self-Advertised Also, Capacity Is a Key Resource Currently, 10 Gbps Is Basic Resource Unit In 2012, 100 Gbps Will Be a Basic Programmable Resource Unit 100 Gbps Not Merely As A capability For Aggregation of Multiple Small Flows, But As a Single Programmable Resource, e.g., For a Single Stream

25 StarWave: A Multi-100 Gbps Facility StarWave, A New Advanced Multi-100 Gbps Facility and Services Will Be Implemented Within the StarLight International/National Communications Exchange Facility StarWave Is Being Funded To Provide Services To Support Large Scale Data Intensive Science Research Initiatives Facilities Components Will Include: An ITU G. 709 v3 Standards Based Optical Switch for WAN Services, Supporting Multiple 100 G Connections An IEEE 802.3ba Standards Based Client Side Switch, Supporting Multiple 100 G Connections, Multiple 10 G Connections Multiple Other Components (e.g., Optical Fiber Interfaces, Measurement Servers, Test Servers

26 DOE Office of Science ESnet ANI 100 Gbps Testbed StarLight

27

28

29 Evaluations/Demonstrations of Gbps IPv4/IPv6 Disk-to-Disk File Transfer Performance Across LANs and WANs

30 CA*net/Ciena/StarLight/iCAIR 100 Gbps Testbed Sept-Oct 2010, Oct Gbps ~850 Miles, 1368 Km Source: CANARIE

31 CVS 2010/11 Terrestrial Demo Content & Applications 100G 100G Connectivity from Ciena Ottawa to Starlight Gigapop in Chicago. Ability to show simultaneous data-flows sourced from major collaborators. Chicago MEN LAB 10 MEN LAB 10 Tile Display Ottawa Montreal Canadian Brain Imaging Network CBRAIN portal & Metadata

32 Selected Ad Hoc Testbeds and Protoypes Ad Hoc Network Science Research Testbeds New Sevices and Technology Experiments Service Prototypes New Architecture Protypes New Science Network Prototypes

33 FILE T-LEX GOLE vlan 2712 FILE C-wave vlan 2711 vlans 2711/2 U Mackenzie Ampath GOLE USP

34 Phospherous

35

36

37 Asia EU StarLight Facility Chicago Ampath

38 GreenLight International GreenLight explores how researchers can take advantage of data centers linked by high-speed networking in an era of carbon-thrifty computing Partnered with Canada s GreenStar network researching a follow the sun/wind distributed computing environment. Recent studies migrating virtual machines to green energy sites indicate that 100 Gb/s networks are far superior to 10 Gb/s to make this transparent.

39 10GE CAVEwave on the National LambdaRail UIC NU NCSA UCSD

40 Summary Transition From Monolithic, Static Services and Infrastructure To Ad Hoc Specialized, Customized, and Even Personalized Resources: Individualized Communication Services Part of Historic Progression From Monolithic To Individualized Environments (Part of the IT DNA) Design, Implementation, Operation of Distributed Network Platforms Integrated With Sophisticated Control Frameworks Addressed via Advanced Programming Methods Provided With Extremely High Capacity Capabilities Will Be Centered At Advanced Communication Exchanges

41

42 Thanks to the NSF, DOE, DARPA Universities, National Labs, International Partners, and Other Supporters