BigData Express: Toward Predictable, Schedulable, and High-performance Data Transfer. Wenji Wu Internet2 Global Summit May 8, 2018
|
|
- Gloria Barker
- 5 years ago
- Views:
Transcription
1 BigData Express: Toward Predictable, Schedulable, and High-performance Data Transfer Wenji Wu Internet2 Global Summit May 8, 2018
2 BigData Express Funded by DOE s office of Advanced Scientific Computing Research (ASCR) Collaborative effort by Fermilab and Oak Ridge National Laboratory KISTI joined as a unfunded partner at 2017 ESnet provides WAN service A three-year research project Start: Oct 1, 2015 End: Sep 30,
3 BigData Express Research Team FNAL Wenji Wu (PI) Qiming Lu Liang Zhang Amy Jin Sajith Sasidharan Phil DeMar ORNL Nageswara Rao Gary Liu KISTI Syed Asif Shah Seo-Young Noh Jin Kim Note: KISTI and ESnet are unfunded project partners
4 DOE Leadership facilities offer computing and Big datacomputing enables scientific discoveries storage resources needed to process and analyze science data BigData Express Goal The efficient movement of science data from their sources into processing and storage facilities and ultimately on to user A distributed middleware system that provides a analysis is critical to the success of any such endeavor. schedulable, predictable, and high-performance transfer for function the DOE s large-scale Datadata transfer is now service an essential for science discoveries, particularly within datacollaborators. environments. science facilities andbig their
5 Why BigData Express? Targeted at optimizing data transfers in high-speed networks Large-scale data movement of Big Data Science High-speed network environments (40/100GE+) Builds on Multicore-Aware Data Transfer Middleware (MDTM) mdtmftp: a high-performance data transfer tool Pipelined I/O-centric design to streamline data transfer MDTM optimizes use of underlying multicore system Extremely efficient in transferring of Lots Of Small Files (LOSF) Orchestrates system (DTN), storage, & network (SDN) resources To provide full end-to-end performance optimization
6 BigData Express versus SENSE BigData Express is data transfer middleware Uses SENSE for WAN SDN services SENSE is a network service Provides higher-level applications with SDN-type services BigData Express is an application to SENSE BigData Express and SENSE are each stand-alone services in their own right BigData Express works fine without SENSE WAN component is simply Best Effort SENSE is agnostic to higher-level applications using its services
7 BigData Express Major Components BDE Web Portal Allow users to access BigData Express data transfer services BDE Scheduler DTN as a service Co-scheduling of DTN, storage, and network BDE AmoebaNet Network as a service mdtmftp a high-performance data transfer engine
8 BigData Express Major Components (cont.) DTN Agents Manage and configure DTNs Collect and report the DTN configuration and status Storage Agents Manage and configure storage systems Data Transfer Launching Agent Launch data transfer jobs Support different data transfer protocols
9 BigData Express -- Distributed A Peer-to-Peer model
10 BigData Express -- Flexible DTNs DTNs Data Transfer Federation Data Transfer Federation Flexible to set up data transfer federations Data Transfer Federation Networks DTNs Providing inherent support for incremental deployment DTNs DTNs
11 BigData Express -- Scalable BDE Web Portal SDN Agent SDN Agent (AmoebaNet) Message Queue BDE Scheduler BDE Scheduler DTN Agent DTN Agent DTN Agent Storage Agent Storage Agent Storage Agent Data Transfer Data Transfer Launching Data Transfer Agent Launching Agent Launching Agent BigData Express scheduler manages site resources through agents Use RabbitMQ as message bus
12 BigData Express -- Extensible BDE Web Portal SDN Agent SDN Agent (AmoebaNet) Message Queue BDE Scheduler BDE Scheduler DTN Agent DTN Agent DTN Agent Data Transfer Launching Agent Data Transfer Launching Agent Data Transfer Launching Agent Storage Agent Storage Agent Storage Agent mdtmftp Plugin GridFTP Plugin SRM Plugin XrootD Plugin Extensible Plugin framework to support various data transfer protocols mdtmftp, GridFTP, SRM, XrootD,
13 BigData Express -- End-to-End Data Transfer Model Site A - Smart E2E Data Transfer Orchestrator Web Portal Scheduler Data Transfer Launching Agent mdtmftp DTN Agent Resource negotiation & brokering AmoebaNet Resource negotiation & brokering SENSE Resource negotiation & brokering AmoebaNet Site B - Smart E2E Data Transfer Orchestrator Web Portal Scheduler Data Transfer Launching Agent mdtmftp DTN Agent Application-aware network service o On-demand programming Fast-provisioning of end-to-end network paths with guaranteed QoS Storage Agent LAN WAN LAN Storage Agent Distributed resource negotiation & brokering Storage Edge DTN Edge DTN Storage A End-to-End Data Transfer Loop with Guaranteed QoS
14 BigData Express -- Three Types of Data Transfer Real-time data transfer Deadline-bound data transfer Best-effort data transfer
15 BigData Express vs. Globus Online Features BigData Express Globus Online Architecture Supported Protocols SDN Support Supported Data Transfers Error Handling Distributed service Flexible to set up data transfer federations Extensible plugin framework to support multiple protocols: o mdtmftp o GridFTP, XrootD, SRM (coming soon) Yes, Network as a service Fast-provisioning end-to-end network paths with guaranteed QoS Real-time data transfer Deadline-bound data transfer Best-effort data transfer Checksum Retransmit Centralized service GridFTP No Best-effort data transfer Checksum Retransmit
16 BigData Express SC 17 DEMO BigData Express: a schedulable, predictable, and high-performance data transfer service QoS-guaranteed data transfer DTN as a service Network as a service Distributed resource brokering/matching A DOE/SC/ASCR-sponsored research project Software is available at:
17 A Cross-Pacific SDN Testbed HP Z91000 AmoebaNet KREONET STP STP 40GE GE BDE Web Protal DTN2 BDE Scheduler StarLight STP GE GE DTN3 DTN ESNET OSCARS KISTI, South Korea KISTI SW 600W Chicago To Internet STP FNAL Border router bde-hp1.fnal.gov ESnet NSI Circuit Service AmoebaNet 40GE 4/1 73 4/3 Pica8 P5101 4/5 4/7 40GE 47 Pica8 P yosemite.fnal.gov 4/2 74 4/4 4/6 4/ BDE Web Protal BDE Scheduler 40GE 40GE 40GE GE 40GE BDE1 BDE2 BDE3 BDE-hp5 BDE4 Lustre file system wwwld1 (mgt) Infiniband Switch wwwld5 (oss) wwwld2 (oss) wwwld3 (oss) wwwld4 (oss) wwwld6 (oss) FNAL, US
18 BigData Express Deployment Completed deployment: KISTI, UMD, StarLight, FNAL Ongoing deployment: KSTAR, ESnet Work with StarLight to deploy BDE at XRPs Pacific Research Platform (PRP) National Research Platform (NRP) Global Research Platform (GRP) The European Research Platform (ERP) Asia Research Platform (ARP) Collaborate with SENSE for BDE+SENSE deployment Work with US CMS to deploy BDE at US CMS sites
19 Fusion community Support Science Work with KSTAR, KISTI, PPPL, and ORNL to transfer/stream data from KSTAR to US research institutions XRPs (PRP, NRP, GRP, ERP, ARP) Work with StarLight to deploy BDE at XRPs to support various science HEP community Work with US CMS to deploy BDE at US CMS sites PI has been invited to give a BDE demo for US CMS Tentatively scheduled for the last week of May, 2018
20 More information about BigData Express PI: Wenji Wu, Fermilab
BigData Express: Toward Predictable, Schedulable, and High-performance Data Transfer. Fermilab, May 2018
BigData Express: Toward Predictable, Schedulable, and High-performance Data Transfer Fermilab, May 2018 BigData Express Research Team FNAL Wenji Wu (PI) Qiming Lu Liang Zhang Amy Jin Sajith Sasidharan
More informationBigData Express: Toward Predictable, Schedulable, and High-Performance Data Transfer. BigData Express Research Team November 10, 2018
BigData Express: Toward Predictable, Schedulable, and High-Performance Data Transfer BigData Express Research Team November 10, 2018 Many people s hard work FNAL: ESnet: icair/starlight: KISTI: Qiming
More informationSCA19 APRP. Update Andrew Howard - Co-Chair APAN APRP Working Group. nci.org.au
SCA19 APRP Update Andrew Howard - Co-Chair APAN APRP Working Group 1 What is a Research Platform Notable Research Platforms APRP History Participants Activities Overview We live in an age of rapidly expanding
More informationSENSE: SDN for End-to-end Networked Science at the Exascale
SENSE: SDN for End-to-end Networked Science at the Exascale SENSE Research Team INDIS Workshop, SC18 November 11, 2018 Dallas, Texas SENSE Team Sponsor Advanced Scientific Computing Research (ASCR) ESnet
More informationIRNC:RXP SDN / SDX Update
30 September 2016 IRNC:RXP SDN / SDX Update John Hess Darrell Newcomb GLIF16, Miami Pacific Wave: Overview Joint project of CENIC (California regional research and
More informationPacific Wave: Building an SDN Exchange
Pacific Wave: Building an SDN Exchange Will Black, CENIC - Pacific Wave Internet2 TechExchange San Francisco, CA Pacific Wave: Overview Joint project between CENIC and PNWGP Open Exchange supporting both
More informationGLIF September 2017 Sydney, Australia Gerben van Malenstein, SURFnet & John Hess, Pacific Wave
AutoGOLE MEICAN Pilot plans for next 5 years SURFnet network upgrade GLIF 2017 25 27 September 2017 Sydney, Australia Gerben van Malenstein, SURFnet & John Hess, Pacific Wave National Research & Education
More informationSLATE. Services Layer at the Edge. First Meeting of the National Research Platform Montana State University August 7-8, 2017
SLATE Services Layer at the Edge Rob Gardner University of Chicago Shawn McKee University of Michigan Joe Breen University of Utah First Meeting of the National Research Platform Montana State University
More informationTowards Network Awareness in LHC Computing
Towards Network Awareness in LHC Computing CMS ALICE CERN Atlas LHCb LHC Run1: Discovery of a New Boson LHC Run2: Beyond the Standard Model Gateway to a New Era Artur Barczyk / Caltech Internet2 Technology
More informationDesign patterns for data-driven research acceleration
Design patterns for data-driven research acceleration Rachana Ananthakrishnan, Kyle Chard, and Ian Foster The University of Chicago and Argonne National Laboratory Contact: rachana@globus.org Introduction
More informationAnalysis of CPU Pinning and Storage Configuration in 100 Gbps Network Data Transfer
Analysis of CPU Pinning and Storage Configuration in 100 Gbps Network Data Transfer International Center for Advanced Internet Research Northwestern University Se-young Yu Jim Chen, Joe Mambretti, Fei
More informationAsia Pacific and European initiatives in HPC and PRP - South Korea
National Institute of Supercomputing and Networking SCAsia2018@Singapore Asia Pacific and European initiatives in HPC and PRP - South Korea Jeonghoon Moon 27 th Mar. 2018 National Institute of Supercomputing
More informationCMS Data Transfer Challenges and Experiences with 40G End Hosts
CMS Data Transfer Challenges and Experiences with 40G End Hosts NEXT Technology Exchange Advanced Networking / Joint Techs Indianapolis, October 2014 Azher Mughal, Dorian Kcira California Institute of
More informationUltraScience Net Update: Network Research Experiments
UltraScience Net Update: Network Research Experiments Nagi Rao, Bill Wing, Susan Hicks, Paul Newman, Steven Carter Oak Ridge National Laboratory raons@ornl.gov https://www.csm.ornl.gov/ultranet February
More informationCarrier SDN for Multilayer Control
Carrier SDN for Multilayer Control Savings and Services Víctor López Technology Specialist, I+D Chris Liou Vice President, Network Strategy Dirk van den Borne Solution Architect, Packet-Optical Integration
More informationN. Marusov, I. Semenov
GRID TECHNOLOGY FOR CONTROLLED FUSION: CONCEPTION OF THE UNIFIED CYBERSPACE AND ITER DATA MANAGEMENT N. Marusov, I. Semenov Project Center ITER (ITER Russian Domestic Agency N.Marusov@ITERRF.RU) Challenges
More informationIntegration of Network Services Interface version 2 with the JUNOS Space SDK
Integration of Network Services Interface version 2 with the JUNOS Space SDK Radosław Krzywania, Michał Balcerkiewicz, Bartosz Belter Poznan Supercomputing and Networking Center, ul. Z. Noskowskiego 12/14,
More informationESnet s (100G) SDN Testbed
ESnet s (100G) SDN Testbed Inder Monga and ESnet SDN team Interna:onal SDN Testbed, March 2015 Outline Testbeds in ESnet Mo:va:on: Building a Scalable SDN WAN testbed Hardware and Deployment Status First
More informationScheduling Computational and Storage Resources on the NRP
Scheduling Computational and Storage Resources on the NRP Rob Gardner Dima Mishin University of Chicago UCSD Second NRP Workshop Montana State University August 6-7, 2018 slides: http://bit.ly/nrp-scheduling
More informationEvolution of the ATLAS PanDA Workload Management System for Exascale Computational Science
Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science T. Maeno, K. De, A. Klimentov, P. Nilsson, D. Oleynik, S. Panitkin, A. Petrosyan, J. Schovancova, A. Vaniachine,
More informationEvent-Based Software-Defined Networking: Build a Secure Science DMZ
White Paper Event-Based Software-Defined Networking: Build a Secure Science DMZ What You Will Learn As the need to efficiently move large data sets around the world increases, the Science DMZ - built at
More informationPublic Cloud Connection for R&E Network. Jin Tanaka APAN-JP/KDDI
Public Cloud Connection for R&E Network Jin Tanaka APAN-JP/KDDI 45th APAN Meeting in Singapore 28th March 2018 Hyper Scale Public cloud and research & science data NASA EOSDIS(Earth Observing System Data
More informationVirtual Circuits Landscape
Virtual Circuits Landscape Summer 2010 ESCC Meeting Columbus, OH Evangelos Chaniotakis, ESnet Network Engineer Lawrence Berkeley National Lab Context and Goals Guaranteed bandwidth services are maturing.
More informationNext Generation Integrated Architecture SDN Ecosystem for LHC and Exascale Science. Harvey Newman, Caltech
Next Generation Integrated Architecture SDN Ecosystem for LHC and Exascale Science Joint Genome Institute LHC Beyond the Higgs Boson LSST SKA Harvey Newman, Caltech NSF CC*/CICI Workshop: Data Integration
More informationA More Realistic Way of Stressing the End-to-end I/O System
A More Realistic Way of Stressing the End-to-end I/O System Verónica G. Vergara Larrea Sarp Oral Dustin Leverman Hai Ah Nam Feiyi Wang James Simmons CUG 2015 April 29, 2015 Chicago, IL ORNL is managed
More informationDWDM-RAM: DARPA-Sponsored Research for Data Intensive Service-on-Demand Advanced Optical Networks
DWDM-RAM: DARPA-Sponsored Research for Intensive -on-demand Advanced Optical Networks DWDM RAM @LIGHTspeed Optical Abundant Bandwidth Meets Grid The Intensive App Challenge: Emerging data intensive applications
More informationSoftware-Defined WAN: Application-centric Virtualization and Visibility
Software-Defined WAN: Application-centric Virtualization and Visibility Dongkyun Kim, KISTI mirr@kisti.re.kr June 23, KRNet2015 Introduction Software-Defined WAN SD-WAN Optimization, Virtualization, Visibility,
More informationZhengyang Liu University of Virginia. Oct 29, 2012
SDCI Net: Collaborative Research: An integrated study of datacenter networking and 100 GigE wide-area networking in support of distributed scientific computing Zhengyang Liu University of Virginia Oct
More informationAutoBAHN Provisioning guaranteed capacity circuits across networks
AutoBAHN Provisioning guaranteed capacity circuits across networks Afrodite Sevasti, GRNET 1 st End-to-end workshop: Establishing lightpaths 1-2 December 2008, TERENA, Amsterdam AutoBAHN is a research
More informationEngagement With Scientific Facilities
Engagement With Scientific Facilities Eli Dart, Network Engineer ESnet Science Engagement Lawrence Berkeley National Laboratory Global Science Engagement Panel Internet2 Technology Exchange San Francisco,
More informationAllowing Users to Run Services at the OLCF with Kubernetes
Allowing Users to Run Services at the OLCF with Kubernetes Jason Kincl Senior HPC Systems Engineer Ryan Adamson Senior HPC Security Engineer This work was supported by the Oak Ridge Leadership Computing
More informationChallenges of Big Data Movement in support of the ESA Copernicus program and global research collaborations
APAN Cloud WG Challenges of Big Data Movement in support of the ESA Copernicus program and global research collaborations Lift off NCI and Copernicus The National Computational Infrastructure (NCI) in
More informationVC3. Virtual Clusters for Community Computation. DOE NGNS PI Meeting September 27-28, 2017
VC3 Virtual Clusters for Community Computation DOE NGNS PI Meeting September 27-28, 2017 Douglas Thain, University of Notre Dame Rob Gardner, University of Chicago John Hover, Brookhaven National Lab A
More informationLong Term Data Preservation for CDF at INFN-CNAF
Long Term Data Preservation for CDF at INFN-CNAF S. Amerio 1, L. Chiarelli 2, L. dell Agnello 3, D. De Girolamo 3, D. Gregori 3, M. Pezzi 3, A. Prosperini 3, P. Ricci 3, F. Rosso 3, and S. Zani 3 1 University
More informationData Replication: Automated move and copy of data. PRACE Advanced Training Course on Data Staging and Data Movement Helsinki, September 10 th 2013
Data Replication: Automated move and copy of data PRACE Advanced Training Course on Data Staging and Data Movement Helsinki, September 10 th 2013 Claudio Cacciari c.cacciari@cineca.it Outline The issue
More informationThe European DataGRID Production Testbed
The European DataGRID Production Testbed Franck Bonnassieux CNRS/UREC ENS-Lyon France DataGrid Network Work Package Manager Franck.Bonnassieux@ens-lyon.fr Presentation outline General DataGrid project
More informationHuawei Agile Controller. Agile Controller
Huawei 1 1 Product Overview Product Features is the latest user-centric and application-based, automatic network resource control system developed by Huawei. This system is positioned as the "Smart Brain"
More informationOpenStack internal messaging at the edge: In-depth evaluation. Ken Giusti Javier Rojas Balderrama Matthieu Simonin
OpenStack internal messaging at the edge: In-depth evaluation Ken Giusti Javier Rojas Balderrama Matthieu Simonin Who s here? Ken Giusti Javier Rojas Balderrama Matthieu Simonin Fog Edge and Massively
More informationSteven Carter. Network Lead, NCCS Oak Ridge National Laboratory OAK RIDGE NATIONAL LABORATORY U. S. DEPARTMENT OF ENERGY 1
Networking the National Leadership Computing Facility Steven Carter Network Lead, NCCS Oak Ridge National Laboratory scarter@ornl.gov 1 Outline Introduction NCCS Network Infrastructure Cray Architecture
More informationDoing Science on GENI GENI and NRP
Doing Science on GENI GENI and NRP Ilya Baldin, ibaldin@renci.org RENCI / UNC Chapel Hill With much help from Paul Ruth (RENCI/UNC), Anirban Mandal (RENCI/UNC), Yufeng Xin (RENCI/UNC) and Jeff Chase (Duke
More informationThe Science DMZ: Evolution
The Science DMZ: Evolution Eli Dart, ESnet CC-NIE PI Meeting Washington, DC May 1, 2014 Why Are We Doing This? It s good to build high-quality infrastructure As network engineers, we like building networks
More informationDWDM-RAM: DARPA-Sponsored Research for Data Intensive Service-on-Demand Advanced Optical Networks
DWDM RAM @LIGHTspeed DWDM-RAM: DARPA-Sponsored Research for Intensive -on-demand Advanced Optical Networks Tal Lavian BUSINESS WITHOUT BOUNDARIES Optical Abundant Bandwidth Meets Grid The Intensive App
More informationProgrammable Information Highway (with no Traffic Jams)
Programmable Information Highway (with no Traffic Jams) Inder Monga Energy Sciences Network Scientific Networking Division Lawrence Berkeley National Lab Exponential Growth ESnet Accepted Traffic: Jan
More information18th WRNP Workshop RNP May Belém, Brasil Gerben van Malenstein
AutoGOLE MEICAN Pilot plans for next 5 years SURFnet network upgrade 18th WRNP Workshop RNP 15-16 May Belém, Brasil Gerben van Malenstein National Research & Education Network of the Netherlands Connecting
More informationOn the EGI Operational Level Agreement Framework
EGI-InSPIRE On the EGI Operational Level Agreement Framework Tiziana Ferrari, EGI.eu EGI Chief Operations Officer 1 Outline EGI and its ecosystem EGI Service Infrastructure Operational level agreements
More informationTitle DC Automation: It s a MARVEL!
Title DC Automation: It s a MARVEL! Name Nikos D. Anagnostatos Position Network Consultant, Network Solutions Division Classification ISO 27001: Public Data Center Evolution 2 Space Hellas - All Rights
More informationHeterogeneous Interconnection between SDN and Layer2 Networks based on NSI
Heterogeneous Interconnection between SDN and Layer2 Networks based on NSI Ta-Yuan Chou, Wun-Yuan Huang, Hui-Lan Lee, Te-Lung Liu, Joe Mambretti*, Jim Hao Chen*, Fei Yeh* National Center for High-Performance
More informationPacificWave Update and Future Directions
PacificWave Update and Future Directions John Silvester University of Southern California Chair, CENIC APAN20, Taipei, 2005.08.24 Overview Introduction to Pacific Wave Current Status including recent additions
More informationMesosphere and the Enterprise: Run Your Applications on Apache Mesos. Steve Wong Open Source Engineer {code} by Dell
Mesosphere and the Enterprise: Run Your Applications on Apache Mesos Steve Wong Open Source Engineer {code} by Dell EMC @cantbewong Open source at Dell EMC {code} by Dell EMC is a group of passionate open
More informationCisco APIC Enterprise Module Simplifies Network Operations
Cisco APIC Enterprise Module Simplifies Network Operations October 2015 Prepared by: Zeus Kerravala Cisco APIC Enterprise Module Simplifies Network Operations by Zeus Kerravala October 2015 º º º º º º
More informationVortex Whitepaper. Simplifying Real-time Information Integration in Industrial Internet of Things (IIoT) Control Systems
Vortex Whitepaper Simplifying Real-time Information Integration in Industrial Internet of Things (IIoT) Control Systems www.adlinktech.com 2017 Table of Contents 1. Introduction........ P 3 2. Iot and
More information* Inter-Cloud Research: Vision
* Inter-Cloud Research: Vision for 2020 Ana Juan Ferrer, ATOS & Cluster Chair Vendor lock-in for existing adopters Issues: Lack of interoperability, regulatory context, SLAs. Inter-Cloud: Hardly automated,
More informationSimplify and automate your network with Cisco DNA
Simplify and automate your network with Cisco DNA Mr. Brink Sanders Managing Director, Software and Network Transformation Cisco Asia Pacific and Japan March, 2017 Agenda Software-Defined Networking (SDN)
More informationGrid Architectural Models
Grid Architectural Models Computational Grids - A computational Grid aggregates the processing power from a distributed collection of systems - This type of Grid is primarily composed of low powered computers
More informationTransferring a Petabyte in a Day. Raj Kettimuthu, Zhengchun Liu, David Wheeler, Ian Foster, Katrin Heitmann, Franck Cappello
Transferring a Petabyte in a Day Raj Kettimuthu, Zhengchun Liu, David Wheeler, Ian Foster, Katrin Heitmann, Franck Cappello Huge amount of data from extreme scale simulations and experiments Systems have
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationData Transfers Between LHC Grid Sites Dorian Kcira
Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference
More informationApplication Centric Microservices Ken Owens, CTO Cisco Intercloud Services. Redhat Summit 2015
Application Centric Microservices Ken Owens, CTO Cisco Intercloud Services Redhat Summit 2015 Agenda Introduction Why Application Centric Application Deployment Options What is Microservices Infrastructure
More informationAdvance Reservation Access Control Using Software-Defined Networking and Tokens
Advance Reservation Access Control Using Software-Defined Networking and Tokens I N N OVAT I N G T H E N E T WORK FO R DATA I N T E N S I V E S C I E N C E ( I N D I S ) 2016 J OAQUI N C HUNG, E U N -
More informationEnabling the Next Generation of SDN
Enabling the Next Generation of SDN Brian O Connor (ONF) brian@opennetworking.org P4 Workshop on June 5, 2018 Link to slides: https://goo.gl/6hfg1h Presenting on behalf of Google and ONF Background Google
More informationChina Unicom SDN Practice in WAN. Lv Chengjin/Ma Jichun, China Unicom
China Unicom SDN Practice in WAN Lv Chengjin/Ma Jichun, China Unicom What Will Operator SDN Do? Two development directions Operator Softwaredefined networking (SDN) 1. Cloudify traditional services 2.
More informationIntroduction to HPC Parallel I/O
Introduction to HPC Parallel I/O Feiyi Wang (Ph.D.) and Sarp Oral (Ph.D.) Technology Integration Group Oak Ridge Leadership Computing ORNL is managed by UT-Battelle for the US Department of Energy Outline
More informationICN for Cloud Networking. Lotfi Benmohamed Advanced Network Technologies Division NIST Information Technology Laboratory
ICN for Cloud Networking Lotfi Benmohamed Advanced Network Technologies Division NIST Information Technology Laboratory Information-Access Dominates Today s Internet is focused on point-to-point communication
More informationSD-WAN. What is it anyway?
SD-WAN What is it anyway? Presenters Moderator: Beth English, EE and Associates Panelists: Nick Olivares, Communications Strategies Tom Brannen, Cloud and Wire What SD-WAN isn t It is not a new type of
More informationImplementation of the Pacific Research Platform over Pacific Wave
Implementation of the Pacific Research Platform over Pacific Wave 21 September 2015 CANS, Chengdu, China Dave Reese (dave@cenic.org) www.pnw-gigapop.net A Brief History of Pacific Wave n Late 1990 s: Exchange
More informationSKA Regional Centre Activities in Australasia
SKA Regional Centre Activities in Australasia Dr Slava Kitaeff CSIRO-ICRAR APSRC Project Engineer ERIDANUS National Project Lead Why SKA Regional Centres? SKA 1 Observatory Compute capacity: 100 Pflops
More informationKREONET-S: Software-Defined Wide Area Network Design and Deployment on KREONET
KREONET-S: Software-Defined Wide Area Network Design and Deployment on KREONET Dongkyun Kim, Yong-Hwan Kim, Chanjin Park, and Kyu-Il Kim Abstract Software-defined networking (SDN) has emerged to achieve
More informationSD-WAN Transform Your Agency
Federal SD-WAN Transform Your Agency 1 Overview Is your agency facing network traffic challenges? Is migration to the secured cloud hogging scarce bandwidth? How about increased mobile computing that is
More informationHP Servers. HP ProLiant DL580 Gen8 Server
HP Servers Today HP is adding new performance, efficiency and reliability features to the industry-leading ProLiant Scaleup server portfolio with the new HP ProLiant DL580 Gen8 Server. Below is a summary
More informationIntroduction to FREE National Resources for Scientific Computing. Dana Brunson. Jeff Pummill
Introduction to FREE National Resources for Scientific Computing Dana Brunson Oklahoma State University High Performance Computing Center Jeff Pummill University of Arkansas High Peformance Computing Center
More informationApache CloudStack. Sebastien Goasguen Open Source Office,
Apache CloudStack Sebastien Goasguen Open Source Office, Citrix @sebgoa IaaS Landscape IaaS is really: A Data Center Orchestrator Data storage Data movement Data processing That can: Handle failures Support
More information5 August 2010 Eric Boyd, Internet2 Deputy CTO
5 August 2010 Eric Boyd, Internet2 Deputy CTO Extending IDC based Dynamic Circuit Networking Services Internet2 and Dynamic Circuit Networking Internet2 has been working with partners on dynamic circuit
More informationSoftware-Defined Networking (SDN) Overview
Reti di Telecomunicazione a.y. 2015-2016 Software-Defined Networking (SDN) Overview Ing. Luca Davoli Ph.D. Student Network Security (NetSec) Laboratory davoli@ce.unipr.it Luca Davoli davoli@ce.unipr.it
More informationRegional e-infrastructures
Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing Regional e-infrastructures Dr. Ognjen Prnjat, GRNET CHAIN-REDS NKN Workshop, Assam www.chain-project.eu
More informationStatus and Trends in Networking at LHC Tier1 Facilities
Journal of Physics: Conference Series Status and Trends in Networking at LHC Tier1 Facilities To cite this article: A Bobyshev et al 2012 J. Phys.: Conf. Ser. 396 042011 View the article online for updates
More informationPhilip C. Roth. Computer Science and Mathematics Division Oak Ridge National Laboratory
Philip C. Roth Computer Science and Mathematics Division Oak Ridge National Laboratory A Tree-Based Overlay Network (TBON) like MRNet provides scalable infrastructure for tools and applications MRNet's
More informationProgress Report. Project title: Resource optimization in hybrid core networks with 100G systems
Progress Report DOE award number: DE-SC0002350 Name of the recipient: University of Virginia Project title: Resource optimization in hybrid core networks with 100G systems Principal investigator: Malathi
More informationAn NDN Testbed for Large-scale Scientific Data
An NDN Testbed for Large-scale Scientific Data Huhnkuk Lim Korea Institute of Science & Technology Information (KISTI) NDNComm 2015 Sep. 28, 2015 Motivations on NDN for Large-scale Scientific Application
More informationProduction Petascale Climate Data Replication at NCI Lustre and our engagement with the Earth Systems Grid Federation (ESGF)
Joseph Antony, Andrew Howard, Jason Andrade, Ben Evans, Claire Trenham, Jingbo Wang Production Petascale Climate Data Replication at NCI Lustre and our engagement with the Earth Systems Grid Federation
More informationKepler and Grid Systems -- Early Efforts --
Distributed Computing in Kepler Lead, Scientific Workflow Automation Technologies Laboratory San Diego Supercomputer Center, (Joint work with Matthew Jones) 6th Biennial Ptolemy Miniconference Berkeley,
More informationVIEWS ON 5G ARCHITECTURE
ETSI SUMMIT ON 5G NETWORK INFRASTRUCTURE VIEWS ON 5G ARCHITECTURE Bernard BARANI, Acting Head of Unit European Commission DG CONNECT Future Connectivity Systems All rights reserved 5G as an EC DSM Priority
More informationEnabling a SuperFacility with Software Defined Networking
Enabling a SuperFacility with Software Defined Networking Shane Canon Tina Declerck, Brent Draney, Jason Lee, David Paul, David Skinner May 2017 CUG 2017-1 - SuperFacility - Defined Combining the capabilities
More informationBNL Dimitrios Katramatos Sushant Sharma Dantong Yu
USC/ISI Tom Lehman Xi Yang ESnet Chin Guok Eric Pouyoul Inder Monga Vangelis Chaniotakis Bharath Ramaprasad (UMass) UNM Nasir Ghani Feng Gu Kaile Liang BNL Dimitrios Katramatos Sushant Sharma Dantong Yu
More informationHuawei Agile Controller. Agile Controller 1
Huawei Agile Controller Agile Controller 1 Agile Controller 1 Product Overview Agile Controller is the latest user- and application-based network resource auto control system offered by Huawei. Following
More informationGÉANT L3VPN Service Description. Multi-point, VPN services for NRENs
GÉANT L3VPN Service Description Multi-point, VPN services for NRENs Issue Date: 1 November 2017 GÉANT L3VPN Overview The GÉANT L3VPN service offers the National Research and Education Networks (NRENs)
More informationThe UniNet Express Lane Services
The UniNet Express Lane Services Assoc. Prof. Vara Varavithya and Peeranon Wattanapong KMUTNB 1 Contents Introduction Problems Motivation Body of Knowledge Contributions Software-Defined Networks Research
More informationONOS Roadmap. September, 2017
ONOS Roadmap September, 2017 distributed core provides high-availability, scalability and performance abstractions & models allow applications to configure and control the network without becoming dependent
More informationAnalytics of Wide-Area Lustre Throughput Using LNet Routers
Analytics of Wide-Area Throughput Using LNet Routers Nagi Rao, Neena Imam, Jesse Hanley, Sarp Oral Oak Ridge National Laboratory User Group Conference LUG 2018 April 24-26, 2018 Argonne National Laboratory
More informationNetwork Services Interface. OGF NSI standards development progress:
Network Services Interface OGF NSI standards development progress: NSI Framework Doc (spr 11) NSI CS v1.0draft (Aug 11) Feed initial implementation NSI CS v1.0 final (Dec 11) NSI CS v2.0draft, NSI Topology
More informationMagellan Project. Jeff Broughton NERSC Systems Department Head October 7, 2009
Magellan Project Jeff Broughton NERSC Systems Department Head October 7, 2009 1 Magellan Background National Energy Research Scientific Computing Center (NERSC) Argonne Leadership Computing Facility (ALCF)
More informationZero to Microservices in 5 minutes using Docker Containers. Mathew Lodge Weaveworks
Zero to Microservices in 5 minutes using Docker Containers Mathew Lodge (@mathewlodge) Weaveworks (@weaveworks) https://www.weave.works/ 2 Going faster with software delivery is now a business issue Software
More informationSupercomputing Asia Singapore March 26-29, 2018
Next Generation Software Defined Services and the Global Research Platform: A Software Defined Distributed Environment For High Performance Large Scale Data Intensive Science Joe Mambretti, Director, (j-mambretti@northwestern.edu)
More informationSDN for Multi-Layer IP & Optical Networks
SDN for Multi-Layer IP & Optical Networks Sterling d Perrin Senior Analyst, Heavy Reading Agenda Definitions for SDN and NFV SDN Drivers and Barriers SDN Use Cases and Applications General Uses Specific
More informationLHC and LSST Use Cases
LHC and LSST Use Cases Depots Network 0 100 200 300 A B C Paul Sheldon & Alan Tackett Vanderbilt University LHC Data Movement and Placement n Model must evolve n Was: Hierarchical, strategic pre- placement
More informationAPI, DEVOPS & MICROSERVICES
API, DEVOPS & MICROSERVICES RAPID. OPEN. SECURE. INNOVATION TOUR 2018 April 26 Singapore 1 2018 Software AG. All rights reserved. For internal use only THE NEW ARCHITECTURAL PARADIGM Microservices Containers
More informationTraffic Optimization for ExaScale Science Applications
Traffic Optimization for ExaScale Science Applications draft-xiang-alto-exascale-network-optimization-00 Q. Xiang 1 H. May Wang 1 H. Newman 2 G. Bernstein 3 A. Mughal 2 J. Balcas 2 1 Tongji/Yale University
More informationOverview. A fact sheet from Feb 2015
A fact sheet from Feb 2015 U.S. Department of Energy Public-Private Partnerships Give the United States an Edge in Manufacturing Federal investment in scientific discovery and technology is vital to maintaining
More informationEuropean Globus Community Forum The Future of Globus in Europe
European Globus Community Forum The Future of Globus in Europe Michael Krieger, RISC Software GmbH Matthias Hofmann, TU Dortmund University Globus usage in Europe Do we need Globus in Europe??? Number
More informationCEER Cyber-Physical Testbeds (a generational leap)
CEER Cyber-Physical Testbeds (a generational leap) CEER: Cyber-Physical Experimentation (testbed operation support) DATA ASSETS CLOUD Customer TESTBED PEOPLE PROVISION LOCAL SCIENCE Other Testbeds Testbed
More informationProject Vision and Mission
Solving End to End connectivity with GMPLS Radek Krzywania, PSNC, Poland radek.krzywania@man.poznan.pl April 4th 2008, Munich NGN meeting Phopshorus project European and Global alliance of partners to
More information