Internet2 Technology Update. Eric Boyd Deputy Technology Officer

Transcription

1 Internet2 Technology Update Eric Boyd Deputy Technology Officer

2 Internet2 Mission and Goals Internet2 Mission Develop and deploy advanced network applications and technologies, accelerating the creation of tomorrowʼs Internet. Internet2 Goals Enable new generation of applications Re-create leading edge R&E network capability Transfer technology and experience to the global production Internet

3 Overview Cyberinfrastructure and Internet2 Building Distributed R&E Systems Technology Update

4 What is Cyberinfrastructure? NSF Vision Paper a diverse set of technologies, facilities, and services and intangibles like design processes and best practices and shared knowledge. A major technological component is software that participates directly in applications and software tools that aid in the development and management of applications. A critical non-technological element is people and organizations that develop and maintain software, operate equipment and software as it is used, and directly assist end-users in the development and use of applications.

5 CI Encompasses Physical Assets, Information, People, and Skills Working as a System Instrumentation Security Security Computation Control Data Generation Analysis Access Control Simulation Management Security and Access Authorization Security Authentication Researcher Program Input Archive Control Security Data Sets Storage Viewing Funding Agencies Retrieval Search Policy and Funding Campuses Help Desk Resource Providers Data Input. Collab Tools Display Tools Human Support Publishing 3D Imaging Display and Visualization Education And Outreach Network Training Security Program Schema Metadata Ontologies Data Directories

6 Support Members to Meet the Challenge Foster Member-led Development of New Technologies Internet2 helps bring together members of the community to spur development of technologies that do not already exist, or are in the initial stage of development. Examples Middleware architectures and software such as Shibboleth End-to-end performance technologies such as perfsonar Dynamic circuit networking control plane technologies and software

7 Support Members to Meet the Challenge Operating Advanced Services that are beyond a Single Institution Efforts that provide the Internet2 community with central services that are not commercially available, and which are not feasible for a single member to provide. Examples Internet2 Network Internet2 Observatory InCommon Federation

8 Support Members to Meet the Challenge Deploying and Encouraging Developmental Services These are pre-production capabilities that may leverage existing services available to the Internet2 community. Examples Internet2 Dynamic Circuit Network perfsonar network performance measurement infrastructure Phoebus research infrastructure

9 Support Members to Meet the Challenge Supporting Community Collaboration in Deploying and Using Technologies Activities in this area includes hosting and supporting workshops, developing online information materials, and community-driven working groups. Examples Workshops focusing on technology deployment in IPv6, multicast, network performance, dynamic provisioning of circuits and middleware. User communities targeted for deployment of such technologies include K20, the Arts & Humanities, the health sciences, the LHC community, and other scientific communities.

10 Support Members to Meet the Challenge Partnering and Coordinating with Other Organizations Engaged in Cyberinfrastructure For those aspects of cyberinfrastructure in which it is not directly involved, and to extend the reach and ensure coordination of its activities, Internet2 partners with other organizations. Examples TeraGrid and Open Science Grid for distributed computing resources DANTE, RNP, and ESnet for network performance measurement and dynamic provisioning of circuits

11 Cyberinfrastructure Vision Cyberinfrastructure encompasses a comprehensive and integrated view of information technologies These technologies are increasingly critical to the research and education community which Internet2 and its members serve. Developing and implementing effective cyberinfrastructure requires taking a holistic view, as well as working and coordinating with many other organizations. Internet2 works with members and partner organizations to support meeting the challenge of cyberinfrastructure by:

12 Cyberinfrastructure Vision: More Than High-End Computing and Connectivity Focused making greater capabilities available across the science and engineering research communities Allows applications to interoperate across institutions and disciplines Ensures that data and software are preserved and easily available to all Empowers enhanced collaboration over distance and across disciplines Report of the National Science Foundation Blue-Ribbon Advisory Panel on Cyberinfrastructure

13 Supporting Research with Networks Research communities are increasingly distributed across the globe A good deal of collaboration is done through sharing large amounts of data via networks Science applications must operate in an environment of multiple, heterogeneous networks

14 Overview Cyberinfrastructure and Internet2 Building Distributed R&E Systems Technology Update

15 Distributed System Design Goals Take existing scientific applications, without recompilation or awareness of circuits, e.g. Bulk File Transfer Real Time Video Exploit performance possibilities of new networking technologies Preserve current politics of business, to some extent (donʼt upset the apple cart) Improve efficiency of problem diagnosis (eliminate reliance on old boy network )

16 Distributed Systems for Networks To build next generation networks, we need distributed software systems on top of the network hardware Session Layer Abstraction (Phoebus, Community-specific abstraction applications) Dynamic Circuit Networks (DCN, e.g. Internet2 DCN, ESnet SDN, GÉANT2 Autobahn) Performance Measurement Framework (e.g. perfsonar) Information Services (IS) Discovery Topology Authentication, Authorization, and Accounting (AAA, e.g. Shibboleth, etc.)

17 Distributed System Requirements These distributed systems share common requirements: Heterogeneous network architecture Multiple administrative entities; no central authority Local customization of operational environment Applications driven by orthogonal virtual organizations Suggests parallel design approach Toolkit approach Web services / defined APIs

18 Multi-Layer Distributed System Design is parallel for each system Hierarchical dependency relationship Suggests abstracting common components, callback architecture across boundaries

19 Multi-Layer Distributed System Scientific Application Diagnostic Analysis and Visualization Tools Hardware Software/ Servers Session-Layer Abstraction Control Plane Framework Performance Monitoring Layer 1 Information Services Layer 2 Layer 3 AAA Design is parallel for each system Hierarchical dependency relationship Suggests abstracting common components, callback architecture across boundaries Creates a common network abstraction toolkit to present to application

20 Importance of Standards Note the importance of standards Between distributed software systems Between discipline community application and network abstraction toolkit Assumes standards-compatible components of network abstraction toolkit deployed across autonomous networks Suggests need for reference implementation(s)

21 Overview Cyberinfrastructure and Internet2 Building Distributed R&E Systems Technology Update

22 Technology Update Middleware Performance Carla Hunt, MCNC DCN Dale Finkelson Phoebus Bill Owens

23 Middleware Infrastructure Focus: Inter-institutional collaboration Scalable authenticated/authorized access to remote resources Internet2 role: Defining/creating architecture: Shibboleth Tools to implement: Shibboleth, Grouper, Signet Infrastructure/Services to scale: InCommon, USHER

24 Shibboleth Lead universities: Brown, Ohio State, Univ of Washington, Duke, USC Adoption: more than 10,000 deployed sites; national federating software for more than 20 countries Commercial adoption: Microsoft, Google, Elsevier, OCLC, etc Current uses: content acquisition, collaboration tools, access to federal applications, access to domain applications

25 Grouper Group management tool Lead institution Univ of Chicago; other key players include Duke, Brown, Georgia Tech, Univ of Penn, Univ of Memphis, Univ of Newcastle Adoption accelerating (~15 operational deployments now) as versions add capabilities, GUI refines, performance improves Under consideration in some key commercial services

26 Comanage Collaboration management platform externalize identity management from collaboration applications Supports collaborations, virtual organizations, enterprises. VO service centers, etc Lead institution Stanford, with MIT support Leverages Shib, Grouper, Signet Early versions, but adoption rapid LIGO, OOI, campuses

27 Technology Update Middleware Performance Carla Hunt, MCNC DCN Dale Finkelson Phoebus Bill Owens

28 Performance Tools BWCTL Throughput Tests v1.3 rc1 released (April, 2008) OWAMP One Way Latency Tests NDT Common Last Mile Problem Diagnosis NP Toolkit Bundle of tools; build a measurement point with a CD

29 Performance Measurement Framework perfsonar is a consortium of partners (Internet2, Esnet, GEANT2, RNP) perfsonar is a set of protocols for monitoring interoperability perfsonar is a set of software implementing a performance measurement framework

30 perfsonar-ps perfsonar-ps is a specific software package, written entirely in perl, designed for ease of installation & use Developed by Internet2, Esnet, Fermi, SLAC, and University of Delaware Targeted for Internet2 backbone, regional networks, campus networks, and virtual organizations (Vos)

31 perfsonar-ps Beta Release (0.09) (4/15/08) Focus on development of major perfsonar components Name Perl CPAN module LS perfsonar_ps::services::ls::ls SNMP MA perfsonar_ps::services::ma::snmp Status MA perfsonar_ps::services::ma::status CircuitStatus MA perfsonar_ps::services::ma::circuitstatus Topology MA perfsonar_ps::services::ma::topology PingER perfsonar_ps::services::mp::pinger perfsonar-buoy perfsonar_ps::services::ma::perfsonarbuoy

32 Weather Map (Internet2)

33 Gmaps (SLAC)

34 Technology Update Middleware Performance Carla Hunt, MCNC DCN Dale Finkelson Phoebus Bill Owens

35 DCN Global Network

36 Internet2 DCN Provisioning 1-A A A-6-1-1

37 DCN Connectors CENIC Great Plains Network (GPN) MAX Merit NOX Nysernet Pacific Northwest Gigapop (PNWGP) 5 more in discussions

38 DCN Virtual Organization Participation LHC Atlas LHC CMS LIGO 2 others in early discussion

39 DCN Applications Terapaths LamdaStation Phoebus 3 others in early discussion

40 DCN Software Suite OSCARS (IDC) Open source project maintained by Internet2 and ESNet DRAGON (DC) NSF-funded Open source project maintained by MAX, USC ISI EAST, and George Mason University

41 DCN Software Suite Version 0.3 of DCNSS released April 18, Enables Dynamic Provisioning Across Internet2 DCN Web Service (and Web Page) User Interface Web Service Network to Network Interface (for multidomain provisioning) Ethernet Ports/VLANs mapped in to variable capacity SONET circuits RONS or other networks can deploy DCNSS to create their own dynamic network on various technologies

42 Technology Update Middleware Performance Carla Hunt, MCNC DCN Dale Finkelson Phoebus Bill Owens

43 The State of the Net High Latency due to distance High Loss due to shared infrastructure

44 Phoebus in Action Phoebus is based on the concept of creating a unique datamoving session for each application Each time an application is run, specific adaptation points in the backbone known as Phoebus Gateways - are utilized to determine the best, highest performance path For example, a file transfer application may traditionally use the IP network. Once the application is set in motion, Phoebus determines the best network path from end to end for this specific application which could include a combination of IP, DC or other future service. Since the intelligence is in the core of the network, Phoebus enables all types of applications to leverage improved network performance with little to no modification by the end-user The Phoebus model is applicable to future applications as well and may prove to be a factor in the evolution of data transport technology

45 The Phoebus Model Phoebus is a framework and protocol for highperformance networks Phoebus works to transparently split the end-to-end network path into distinct segments Adaptation points are typically chosen at the ingress and egress points of the backbone This minimizes the negative effects of high latency and packet loss on data transfer By localizing their effects By allocating dedicated resources to mitigate the issues

46 The Phoebus Model - Con t Transparent adaptation for existing applications Perform well to nearest Phoebus Gateway and allow the system to do the rest No modification necessary for most applications The Phoebus system has the ability to optimize the performance with a variety of techniques and insights into the state of the network

47 Phoebus-Enabled DC Network End-to-End Session DC Network

48 Deployment Plans - co-deployed with Routers Phoebus nodes at router POPs

49 Deployment Plans - co-deployed with Routers Phoebus nodes at router POPs Internet2 Phoebus Gateway

50