WP7. Mark Kettenis April 18, Joint Institute for VLBI in Europe

Size: px
Start display at page:

Download "WP7. Mark Kettenis April 18, Joint Institute for VLBI in Europe"

Transcription

1 WP7 Mark Kettenis Joint Institute for VLBI in Europe April 18, 2011 Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

2 Goals Create an automated, distributed correlator using the global, shared infrastructure of the EVN and its associated global partners in order to Enable scienctist to use unused telescope time to do, for example, surveys and astrometric monitoring programmes. realizing that Resources are scarce (staff time, disk pool, correlator time) Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

3 How do we realize that? e-vlbi: No impact on disk resources Software correlation: No impact on hardware correlator No/less need for correlator operators Workflow manager: Limit impact on telescope operators Automate things as much of possible......but always require proper authorization first Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

4 Partners JIVE SFXC correlator; Integration with correlator operations OSO Integration with telescope operations; Science input PSNC Workflow manager & Resource broker; Middleware VENT Workflow manager & Resource broker CSIRO/Curtin Global correlation; DiFX correlator Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

5 Radio-astronomy Single Dish Resolution 1/D (D = dish diameter) Limited to 100m Interferometry Resolution 1/D (D = distance between telescopes) Correlate all pairs of telescopes Limited to 60km VLBI Limited to 12000km Even more if you go into space! Individual telescopes with high-resolution clocks Mark Kettenis kettenis@jive.nl (JIVE) NEXPReS WP7 April 18, / 16

6 VLBI Data recorded at telescopes on diskpacks Disks are shipped to correlator: Currently dedicated ASIC-based hardware Software correlator for more flexibility Correlated data is made available to PI Whole process takes weeks to months! Solution: Send data over network: e-vlbi Today: 1Gbps per telescope Next year: 4Gbps per telescope Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

7 EXPReS FP6 programme; Connected telescopes in Europe and beyond Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

8 Cygnus X-3 Tudose et al., 2006 Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

9 EXPReS (How not do do things) Throughput: Disks are slow (especially read + write) GridFTP is slow (compared to UDP-based e-vlbi data transfers) Latency: Complete chunks need to be written before transfers can start Complete chunks need to be transferred before jobs can be submitted Jobs that require significant resources take time to get scheduled Mark Kettenis kettenis@jive.nl (JIVE) NEXPReS WP7 April 18, / 16

10 Experience from EXPReS WebServices Interoperability is an issue Careful evaluation of software tools beforehand Move to a web-based workflow manager Move to JSON-based RPC interface Some MPI implementations work better than others Something we may have to live with Distribution of computation based on time-chunks unusuccesful Uncorrelated data shouldn t hit disk Mark Kettenis kettenis@jive.nl (JIVE) NEXPReS WP7 April 18, / 16

11 VDIF Important innovation; no longer mixing bits of seperate sidebands in the same computer word. Allows separation of sidebands at the source, sending them to multiple correlators running in parallel. Please don t implement legacy formats unless you absolutely have to. Mark Kettenis kettenis@jive.nl (JIVE) NEXPReS WP7 April 18, / 16

12 SFXC Descendant of Huygens correlator Developed at JIVE (Pogrebenko/Oerlemans/Kruithof/Keimpema/Kettenis) Pulsar binning/gating Supports Mark4, VLBA, Mark5B, VDIF (Mark5C) So far only used at JIVE Data product compatible with EVN MkIV VEX-driven + simple configuration file GPL licensed Optionally uses IPP (20% more performance on Intel CPUs) In use for production alongside MarkIV hardware correlator Mark Kettenis kettenis@jive.nl (JIVE) NEXPReS WP7 April 18, / 16

13 DiFX Developed at Swinburne (Deller/Tingay/Brisken/Philips/...) Pulsar binning/gating Multiple phase centers Spectral zoom mode Supports Mark4, VLBA, Mark5B, VDIF (Mark5C) Widely used (LBA, VLBA, Bonn) Latest version (2.0) probably faster than SFXC Data product compatible with VLBA hardware correlator VEX-driven, more complicated configuration file Needs license agreement; Free for Astronomy usage Requires IPP Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

14 Why two correlators? Correlators have different strengths: Horses for Courses Supporting two correlators should make Workflow manager/broker correlator agnostic Easy to adapt for similar applications: software spectrometer (space science)? LEAP? Middleware usable for other real-time stream processing as well Applications outside of radio astronomy! Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

15 Global Correlation Mark Kettenis (JIVE) NEXPReS WP7 April 18, / 16

16 NEXPReS is an Integrated Infrastructure Initiative (I3), funded under the European Union Seventh Framework Programme (FP7/ ) under grant agreement N o RI This presentation reflects only the author s views and that the Union is not liable for any use that may be made of the information contained therein. Mark Kettenis kettenis@jive.nl (JIVE) NEXPReS WP7 April 18, / 16

Real-Time E-VLBI with the SFXC Software Correlator. Mark Kettenis Aard Keimpema JIVE

Real-Time E-VLBI with the SFXC Software Correlator. Mark Kettenis Aard Keimpema JIVE Real-Time E-VLBI with the SFXC Software Correlator Mark Kettenis kettenis@jive.nl Aard Keimpema keimpema@jive.nl JIVE SFXC SFXC = S? FX Correlator Where S? used to mean Super Uses MPI to distribute/parallelize

More information

PoS(11th EVN Symposium)112

PoS(11th EVN Symposium)112 Arpad Szomoru 1 Joint Institute for VLBI in Europe P.O. Box 2, 7990 AA Dwingeloo, the Netherlands E-mail: szomoru@jive.nl Paul Boven Joint Institute for VLBI in Europe P.O. Box 2, 7990 AA Dwingeloo, the

More information

Report on integration of E-VLBI System

Report on integration of E-VLBI System Express Production Real-time e-vlbi Service EXPReS is funded by the European Commission (DG-INFSO), Sixth Framework Programme, Contract #026642 Report on integration of E-VLBI System Title: Report on integration

More information

European VLBI Network

European VLBI Network European VLBI Network Cormac Reynolds, JIVE European Radio Interferometry School, Bonn 12 Sept. 2007 EVN Array 15 dissimilar telescopes Observes 3 times a year (approx 60 days per year) Includes some

More information

D Demonstration of integrated BoD testing and validation

D Demonstration of integrated BoD testing and validation D6.06 - Demonstration of integrated BoD testing and validation Deliverable Number 06 WP # 6 Date 2012-12-30 Authors Jimmy Cullen, The University of Manchester Fedde Bloemhof, JIVE Paul Boven, JIVE Ralph

More information

Grid Integration of Future Arrays of Broadband Radio Telescopes moving towards e-vlbi

Grid Integration of Future Arrays of Broadband Radio Telescopes moving towards e-vlbi Grid Integration of Future Arrays of Broadband Radio Telescopes moving towards e-vlbi Marcin Okoń Poznań Supercomputing and Networking Center, Supercomputing Department INGRID 2007 S. Margherita Ligure,

More information

LBA evlbi Developments. Chris Phillips evlbi Project Scientist 14 September 2007

LBA evlbi Developments. Chris Phillips evlbi Project Scientist 14 September 2007 LBA evlbi Developments Chris Phillips evlbi Project Scientist 14 September 2007 Summary First science paper Offering via normal call for proposals Partner in EXPReS Fringes to Sh as part of APAN Also Da,

More information

The Network Infrastructure for Radio Astronomy in Italy

The Network Infrastructure for Radio Astronomy in Italy The Network Infrastructure for Radio Astronomy in Italy 1 INAF Istituto di Radioastronomia Via Gobetti 101, Bologna, Italy E-mail:m.nanni@ira.inaf.it Franco Mantovani INAF Istituto di Radioastronomia Via

More information

VLBI progress Down-under. Tasso Tzioumis Australia Telescope National Facility (ATNF) 25 September 2008

VLBI progress Down-under. Tasso Tzioumis Australia Telescope National Facility (ATNF) 25 September 2008 VLBI progress Down-under Tasso Tzioumis Australia Telescope National Facility (ATNF) 25 September 2008 Outline Down-under == Southern hemisphere VLBI in Australia (LBA) Progress in the last few years Disks

More information

Radio astronomy data reduction at the Institute of Radio Astronomy

Radio astronomy data reduction at the Institute of Radio Astronomy Mem. S.A.It. Suppl. Vol. 13, 79 c SAIt 2009 Memorie della Supplementi Radio astronomy data reduction at the Institute of Radio Astronomy J.S. Morgan INAF Istituto di Radioastronomia, Via P. Gobetti, 101

More information

IVTW - Auckland

IVTW - Auckland formatter recorder MarkIV VLBA RS422 Mark5A DBBC VSI/H FiLa10G ethernet Mark5C Mark6 FlexBuff RS422 MarkIV VLBA Analog VSI/H DBBC[2] R1002 CDAS ADS[123]000 Digital Ethernet DBBC[2]+FiLa10G R[2]DBE CDAS2

More information

Research Infrastructures Use and Further Needed Development

Research Infrastructures Use and Further Needed Development Joint : Testing and Research Infrastructure for Future Power Grids Research Infrastructures Use and Further Needed Development Mihai Calin The research leading to these results has received funding from

More information

EXPReS SA1: at the half-way mark. Arpad Szomoru JIVE

EXPReS SA1: at the half-way mark. Arpad Szomoru JIVE EXPReS SA1: at the half-way mark Arpad Szomoru JIVE Outline Progress in 2007 Going intercontinental The next 18 months Slide #2 e-vlbi science/test runs 17 e-vlbi science projects accepted since 2006:

More information

Deliverable 7.3. Workflow Manager. Poznao Supercomputing and Networking Center

Deliverable 7.3. Workflow Manager. Poznao Supercomputing and Networking Center Deliverable 7.3 Workflow Manager Poznao Supercomputing and Networking Center Table of contents 1 Introduction... 3 2 E-VLBI Correlation Platform... 3 2.1 Liferay introduction... 3 2.2 NEXPReS model...

More information

PoS(EXPReS09)023. e-evn progress. 3 years of EXPReS. Arpad Szomoru 1

PoS(EXPReS09)023. e-evn progress. 3 years of EXPReS. Arpad Szomoru 1 3 years of EXPReS 1 Joint Institute for VLBI in Europe P.O. Box 2, 7990 AA Dwingeloo, The Netherlands E-mail: szomoru@jive.nl During the past years e-vlbi has undergone a remarkable development. At the

More information

EVN/JIVE Technical developments. Arpad Szomoru, JIVE

EVN/JIVE Technical developments. Arpad Szomoru, JIVE EVN/JIVE Technical developments Arpad Szomoru, JIVE Correlator upgrades New(ish) data output platform Solaris AMD server with large raid array Fully interchangeable with control computers Re-circulation

More information

PoS(10th EVN Symposium)098

PoS(10th EVN Symposium)098 1 Joint Institute for VLBI in Europe P.O. Box 2, 7990 AA Dwingeloo, The Netherlands E-mail: szomoru@jive.nl The, a Joint Research Activity in the RadioNet FP7 2 programme, has as its aim the creation of

More information

NEXPReS WP8 Progress Meeting

NEXPReS WP8 Progress Meeting NEXPReS WP8 Progress Meeting Preparations for Multi Site Testing Ari Mujunen Aalto University Metsähovi Radio Observatory VLBI2SKA Workshop in Aveiro, Portugal, 24-26.5.2011 Research leading to these results

More information

PoS(EXPReS09)036. e-vlbi Networking Tricks. Paul Boven, for the EXPReS team Joint Institute for VLBI in Europe (JIVE)

PoS(EXPReS09)036. e-vlbi Networking Tricks. Paul Boven, for the EXPReS team Joint Institute for VLBI in Europe (JIVE) , for the EXPReS team Joint Institute for VLBI in Europe (JIVE) E-mail: boven@jive.nl Real-time e-vlbi is made possible by having access to long-distance, high-performance networks - straight from the

More information

Guifré Molera, J. Ritakari & J. Wagner Metsähovi Radio Observatory, TKK - Finland

Guifré Molera, J. Ritakari & J. Wagner Metsähovi Radio Observatory, TKK - Finland European 4-Gbps VLBI and evlbi Click to edit Master subtitle style Guifré Molera, J. Ritakari & J. Wagner Observatory, TKK - Finland 1 Outline Introduction Goals for 4 Gbps Testing Results Product Conclusions

More information

Lessons from EXPReS and the future of e-vlbi. 2nd Terena E2E Workshop, Amsterdam,

Lessons from EXPReS and the future of e-vlbi. 2nd Terena E2E Workshop, Amsterdam, Lessons from EXPReS and the future of e-vlbi 2nd Terena E2E Workshop, Amsterdam, 2009-12-08 What is JIVE? Operate the EVN correlator and support astronomers doing VLBI. A collaboration of the major radioastronomical

More information

User Interaction and Workflow Management in Grid enabled e-vlbi Experiments

User Interaction and Workflow Management in Grid enabled e-vlbi Experiments COMPUTATIONAL METHODS IN SCIENCE AND TECHNOLOGY 15(1), 95-104 (2009) User Interaction and Workflow Management in Grid enabled e-vlbi Experiments Ł. Dolata, M. Okoń, D. Stokłosa, Sz. Trocha, N. Meyer, M.

More information

Status of the Bonn Correlator MPIfR BKG correlator. Walter Alef Helge Rottmann

Status of the Bonn Correlator MPIfR BKG correlator. Walter Alef Helge Rottmann Status of the Bonn Correlator MPIfR BKG correlator Walter Alef Helge Rottmann Bonn correlator centre MK IV hardware correlator (XF) 15 Mark 5s: 8 x 5A, 4 x 5B, (1 x 5B+, 2 x 5C) Up to 1 Gbit/s data rate

More information

MISTRAL WORKSHOP #1 Trends on future train-to-wayside communications Barcelona, 07/07/2017

MISTRAL WORKSHOP #1 Trends on future train-to-wayside communications Barcelona, 07/07/2017 MISTRAL WORKSHOP #1 Trends on future train-to-wayside communications Barcelona, 07/07/2017 INDEX Introduction: MISTRAL Project Future technological trends for train-to-wayside communications Future economic

More information

ORIENT/ORIENTplus - Connecting Academic Networks in China and Europe. Jennifer(Jie) An CERNET Center/Tsinghua University 14 Feb.

ORIENT/ORIENTplus - Connecting Academic Networks in China and Europe. Jennifer(Jie) An CERNET Center/Tsinghua University 14 Feb. ORIENT/ORIENTplus - Connecting Academic Networks in China and Europe Jennifer(Jie) An CERNET Center/Tsinghua University 14 Feb.2012 Connecting Academic Networks in China and Europe ORIENT project provides

More information

Long Baseline Array Status

Long Baseline Array Status Long Baseline Array Status Cormac Reynolds, Chris Phillips + LBA Team 19 November 2015 CSIRO ASTRONOMY & SPACE SCIENCE LBA LBA VLBI array operated by CSIRO University of Tasmania, Auckland University of

More information

SLHC-PP DELIVERABLE REPORT EU DELIVERABLE: Document identifier: SLHC-PP-D v1.1. End of Month 03 (June 2008) 30/06/2008

SLHC-PP DELIVERABLE REPORT EU DELIVERABLE: Document identifier: SLHC-PP-D v1.1. End of Month 03 (June 2008) 30/06/2008 SLHC-PP DELIVERABLE REPORT EU DELIVERABLE: 1.2.1 Document identifier: Contractual Date of Delivery to the EC Actual Date of Delivery to the EC End of Month 03 (June 2008) 30/06/2008 Document date: 27/06/2008

More information

jive5ab The Swiss Army knife of (e-)vlbi Harro Verkouter

jive5ab The Swiss Army knife of (e-)vlbi Harro Verkouter jive5ab The Swiss Army knife of (e-)vlbi Harro Verkouter Archaelogy $> cd ~/src/jive5ab $> grep 'int main(' *cc test.cc:int main(int argc, char** argv) { Archaelogy aug 2007 256Mbps e-vlbi w/ China demo

More information

APAN Global Collaboration Linking the World with Light

APAN Global Collaboration Linking the World with Light APAN Global Collaboration Linking the World with Light 1 Andrew Howard Coordinating Engineer Advanced Communications Services and International Development Massive increase in International connectivity

More information

Concept Design of a Software Correlator for future ALMA. Jongsoo Kim Korea Astronomy and Space Science Institute

Concept Design of a Software Correlator for future ALMA. Jongsoo Kim Korea Astronomy and Space Science Institute Concept Design of a Software Correlator for future ALMA Jongsoo Kim Korea Astronomy and Space Science Institute Technologies for Correlators ASIC (Application-Specific Integrated Circuit) e.g, ALMA 64-antenna

More information

Introduction to perfsonar. RIPE SEE5, Tirana, Albania Szymon Trocha Poznań Supercomputing and Networking Center, Poland April 2016

Introduction to perfsonar. RIPE SEE5, Tirana, Albania Szymon Trocha Poznań Supercomputing and Networking Center, Poland April 2016 Introduction to perfsonar RIPE SEE5, Tirana, Albania Szymon Trocha Poznań Supercomputing and ing Center, Poland 19 20 April 2016 Agenda performance problems What is perfsonar perfsonar use cases Deployment

More information

ECMWF s Next Generation IO for the IFS Model

ECMWF s Next Generation IO for the IFS Model ECMWF s Next Generation IO for the Model Part of ECMWF s Scalability Programme Tiago Quintino, B. Raoult, P. Bauer ECMWF tiago.quintino@ecmwf.int ECMWF January 14, 2016 ECMWF s HPC Targets What do we do?

More information

Path to Exascale? Intel in Research and HPC 2012

Path to Exascale? Intel in Research and HPC 2012 Path to Exascale? Intel in Research and HPC 2012 Intel s Investment in Manufacturing New Capacity for 14nm and Beyond D1X Oregon Development Fab Fab 42 Arizona High Volume Fab 22nm Fab Upgrades D1D Oregon

More information

The UniBoard. a RadioNet FP7 Joint Research Activity. Arpad Szomoru, JIVE

The UniBoard. a RadioNet FP7 Joint Research Activity. Arpad Szomoru, JIVE The UniBoard a RadioNet FP7 Joint Research Activity Arpad Szomoru, JIVE Overview Background, project setup Current state UniBoard as SKA phase 1 correlator/beam former Future: UniBoard 2 The aim Creation

More information

Interoperability in Aerospace Public Use Case of CRYSTAL project

Interoperability in Aerospace Public Use Case of CRYSTAL project Interoperability in Aerospace Public Use Case of CRYSTAL project December 3 rd, 2013. Francesco Brunetti, Politecnico di Torino Summary CRYSTAL Overview; CRYSTAL WP2.08: Public Use Case; Public Use Case,

More information

1. Publishable Summary

1. Publishable Summary 1. Publishable Summary 1.1Project objectives and context Identity management (IdM) has emerged as a promising technology to distribute identity information across security domains. In e-business scenarios,

More information

GÉANT Support for Research Within and Beyond Europe

GÉANT Support for Research Within and Beyond Europe GÉANT Support for Research Within and Beyond Europe TNC 2003, Zagreb 19-22 May Marian Garcia, Operations Manager DANTE AGENDA GÉANT Global Research Connectivity Europe, North America and other regions

More information

A Breakthrough in Non-Volatile Memory Technology FUJITSU LIMITED

A Breakthrough in Non-Volatile Memory Technology FUJITSU LIMITED A Breakthrough in Non-Volatile Memory Technology & 0 2018 FUJITSU LIMITED IT needs to accelerate time-to-market Situation: End users and applications need instant access to data to progress faster and

More information

ASTRONOMY & PARTICLE PHYSICS CLUSTER

ASTRONOMY & PARTICLE PHYSICS CLUSTER ASTRONOMY & PARTICLE PHYSICS CLUSTER Giovanni LAMANNA LAPP, Laboratoire d Annecy de Physique des Particules CNRS London, 30 January 2019 ESCAPE - The European Science Cluster of Astronomy & Particle Physics

More information

The Digitising European Industry strategy & H2020 calls related to Cyber-Physical Systems

The Digitising European Industry strategy & H2020 calls related to Cyber-Physical Systems The Digitising European Industry strategy & H2020 calls related to Cyber-Physical Systems #DigitiseEU Dr. Werner Steinhögl European Commission - DG CONNECT Technologies and Systems for Digitising Industry

More information

New Generation Open Content Delivery Networks

New Generation Open Content Delivery Networks Open ContEnt Aware Networks New Generation Open Content Delivery Networks Yannick Le Louédec Orange Labs Workshop Future Media Distribution. November 10 th, 2011 www.ict-ocean.eu The research leading to

More information

GÉANT Mission and Services

GÉANT Mission and Services GÉANT Mission and Services Vincenzo Capone Senior Technical Business Development Officer CREMLIN WP2 Workshop on Big Data Management 15 February 2017, Moscow GÉANT - Networks Manages research & education

More information

VLBI Participation in Next Generation Network Development

VLBI Participation in Next Generation Network Development VLBI Participation in Next Generation Network Development evlbi Workshop Haystack Observatory 9 th April 2002 Steve Parsley parsley@jive.nl Agenda evlbi Network requirements Existing networks and future

More information

TimeServerVlbi - A VLBI data stream real-time monitor

TimeServerVlbi - A VLBI data stream real-time monitor TimeServerVlbi - A VLBI data stream real-time monitor Matteo Stagni March 25, 2015 IRA 485/15 Abstract A couple of Python scripts that serve the purpose of monitoring VLBI data streams and check their

More information

SpaceWire-RT Update. EU FP7 Project Russian and European Partners. SUAI, SubMicron, ELVEES University of Dundee, Astrium GmbH

SpaceWire-RT Update. EU FP7 Project Russian and European Partners. SUAI, SubMicron, ELVEES University of Dundee, Astrium GmbH SpaceWire-RT Update EU FP7 Project Russian and European Partners SUAI, SubMicron, ELVEES University of Dundee, Astrium GmbH 1 Contents SpaceWire-RT project SpaceWire-RT protocols Oversampled SpaceFibre

More information

AutoBAHN Provisioning guaranteed capacity circuits across networks

AutoBAHN Provisioning guaranteed capacity circuits across networks AutoBAHN Provisioning guaranteed capacity circuits across networks Afrodite Sevasti, GRNET 1 st End-to-end workshop: Establishing lightpaths 1-2 December 2008, TERENA, Amsterdam AutoBAHN is a research

More information

Research Infrastructures and Horizon 2020

Research Infrastructures and Horizon 2020 Research Infrastructures and Horizon 2020 Christos VASILAKOS DG Research & 1 st CoPoRI Workshop on EoE 11-12 June 2012 Hamburg, DE The EU Framework Programme for Research and 2014-2020 Research and Europe

More information

An Intelligent Service Oriented Infrastructure supporting Real-time Applications

An Intelligent Service Oriented Infrastructure supporting Real-time Applications An Intelligent Service Oriented Infrastructure supporting Real-time Applications Future Network Technologies Workshop 10-11 -ETSI, Sophia Antipolis,France Karsten Oberle, Alcatel-Lucent Bell Labs Karsten.Oberle@alcatel-lucent.com

More information

PROJECT FINAL REPORT. Tel: Fax:

PROJECT FINAL REPORT. Tel: Fax: PROJECT FINAL REPORT Grant Agreement number: 262023 Project acronym: EURO-BIOIMAGING Project title: Euro- BioImaging - Research infrastructure for imaging technologies in biological and biomedical sciences

More information

Test report of transparent local/remote application programming interface for storage elements

Test report of transparent local/remote application programming interface for storage elements NEXPReS is an Integrated Infrastructure Initiative (I3), funded under the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement n RI-261525. Test report of transparent

More information

Data Intensive Science Impact on Networks

Data Intensive Science Impact on Networks Data Intensive Science Impact on Networks Eli Dart, Network Engineer ESnet Network Engineering g Group IEEE Bandwidth Assessment Ad Hoc December 13, 2011 Outline Data intensive science examples Collaboration

More information

An Introduction to Cloud Computing with OpenNebula

An Introduction to Cloud Computing with OpenNebula OpenNebula Cloud Management Platform > Innovative Open Source Technologies and Professional Services to Build your Enterprise-ready Cloud Computing Environment An Introduction to Cloud Computing with OpenNebula

More information

CSCI-GA Operating Systems I/O. Hubertus Franke

CSCI-GA Operating Systems I/O. Hubertus Franke Operating Systems I/O CSCI-GA.2250-001 Hubertus Franke frankeh@cs.nyu.edu External devices that engage in I/O with computer systems can be grouped into three categories: Human readable suitable for communicating

More information

Approaches to I/O Scalability Challenges in the ECMWF Forecasting System

Approaches to I/O Scalability Challenges in the ECMWF Forecasting System Approaches to I/O Scalability Challenges in the ECMWF Forecasting System PASC 16, June 9 2016 Florian Rathgeber, Simon Smart, Tiago Quintino, Baudouin Raoult, Stephan Siemen, Peter Bauer Development Section,

More information

Cisco SAN Analytics and SAN Telemetry Streaming

Cisco SAN Analytics and SAN Telemetry Streaming Cisco SAN Analytics and SAN Telemetry Streaming A deeper look at enterprise storage infrastructure The enterprise storage industry is going through a historic transformation. On one end, deep adoption

More information

CorrelX: A Cloud-Based VLBI Correlator

CorrelX: A Cloud-Based VLBI Correlator CorrelX: A Cloud-Based VLBI Correlator V. Pankratius, A. J. Vazquez, P. Elosegui Massachusetts Institute of Technology Haystack Observatory pankrat@mit.edu, victorpankratius.com 5 th International VLBI

More information

SURFnet network developments 10th E-VLBI workshop 15 Nov Wouter Huisman SURFnet

SURFnet network developments 10th E-VLBI workshop 15 Nov Wouter Huisman SURFnet SURFnet network developments 10th E-VLBI workshop 15 Nov 2011 Wouter Huisman SURFnet Engine for Innovation Mission To improve higher education and research by promoting, developing and operating a trusted,

More information

Today s Agenda. Today s Agenda 9/8/17. Networking and Messaging

Today s Agenda. Today s Agenda 9/8/17. Networking and Messaging CS 686: Special Topics in Big Data Networking and Messaging Lecture 7 Today s Agenda Project 1 Updates Networking topics in Big Data Message formats and serialization techniques CS 686: Big Data 2 Today

More information

D6.7 Demonstration of international BoD connectivity at 4 Gb/s

D6.7 Demonstration of international BoD connectivity at 4 Gb/s D6.7 Demonstration of international BoD connectivity at 4 Gb/s Deliverable Number 07 WP # 6 Date 29th March 2012 Authors Jimmy Cullen, The University of Manchester Ralph Spencer, The University of Manchester

More information

High bandwidth, Long distance. Where is my throughput? Robin Tasker CCLRC, Daresbury Laboratory, UK

High bandwidth, Long distance. Where is my throughput? Robin Tasker CCLRC, Daresbury Laboratory, UK High bandwidth, Long distance. Where is my throughput? Robin Tasker CCLRC, Daresbury Laboratory, UK [r.tasker@dl.ac.uk] DataTAG is a project sponsored by the European Commission - EU Grant IST-2001-32459

More information

MeerKAT Data Architecture. Simon Ratcliffe

MeerKAT Data Architecture. Simon Ratcliffe MeerKAT Data Architecture Simon Ratcliffe MeerKAT Signal Path MeerKAT Data Rates Online System The online system receives raw visibilities from the correlator at a sufficiently high dump rate to facilitate

More information

Research Infrastructures and Horizon 2020

Research Infrastructures and Horizon 2020 Ana Arana Antelo DG Research & Head of Research Infrastructures ERF Workshop - Hamburg, 31 May 2012 Research Infrastructures and Horizon 2020 The EU Framework Programme for Research and 2014-2020 Research

More information

FPGA APPLICATIONS FOR SINGLE DISH ACTIVITY AT MEDICINA RADIOTELESCOPES

FPGA APPLICATIONS FOR SINGLE DISH ACTIVITY AT MEDICINA RADIOTELESCOPES MARCO BARTOLINI - BARTOLINI@IRA.INAF.IT TORINO 18 MAY 2016 WORKSHOP: FPGA APPLICATION IN ASTROPHYSICS FPGA APPLICATIONS FOR SINGLE DISH ACTIVITY AT MEDICINA RADIOTELESCOPES TORINO, 18 MAY 2016, INAF FPGA

More information

Current Status of K5 Software Correlator

Current Status of K5 Software Correlator 3 rd rd e-vlbi Workshop@Makuhari 2004.10.7 Current Status of K5 Software Correlator T.Kondo, Y.Koyama, H.Takeuchi and M. Kimura National Institute of Information and Communications Technology Contents

More information

Data Centres in the Virtual Observatory Age

Data Centres in the Virtual Observatory Age Data Centres in the Virtual Observatory Age David Schade Canadian Astronomy Data Centre A few things I ve learned in the past two days There exist serious efforts at Long-Term Data Preservation Alliance

More information

Jozef Cernak, Marek Kocan, Eva Cernakova (P. J. Safarik University in Kosice, Kosice, Slovak Republic)

Jozef Cernak, Marek Kocan, Eva Cernakova (P. J. Safarik University in Kosice, Kosice, Slovak Republic) ARC tools for revision and nightly functional tests Jozef Cernak, Marek Kocan, Eva Cernakova (P. J. Safarik University in Kosice, Kosice, Slovak Republic) Outline Testing strategy in ARC ARC-EMI testing

More information

Unifying Carrier and Cloud Networks: Problem Statement and Challenges draft-unify-nfvrg-challenges-02

Unifying Carrier and Cloud Networks: Problem Statement and Challenges draft-unify-nfvrg-challenges-02 Unifying Carrier and Cloud Networks: Problem Statement and Challenges draft-unify-nfvrg-challenges-02 Diego Daino (Telecom Italia) A. Csaszar (Ericsson) K. Pentikousis (EICT) M. Kind (Deutsche Telekom

More information

Helix Nebula Science Cloud Pre-Commercial Procurement pilot. 5 April 2016 Bob Jones, CERN

Helix Nebula Science Cloud Pre-Commercial Procurement pilot. 5 April 2016 Bob Jones, CERN Helix Nebula Science Cloud Pre-Commercial Procurement pilot 5 April 2016 Bob Jones, CERN The Helix Nebula Science Cloud public-private partnership LHC computing resources in 2014 3.4 billion CPU hours

More information

Easy Access to Grid Infrastructures

Easy Access to Grid Infrastructures Easy Access to Grid Infrastructures Dr. Harald Kornmayer (NEC Laboratories Europe) On behalf of the g-eclipse consortium WP11 Grid Workshop Grenoble, France 09 th of December 2008 Background in astro particle

More information

The DEEP (and DEEP-ER) projects

The DEEP (and DEEP-ER) projects The DEEP (and DEEP-ER) projects Estela Suarez - Jülich Supercomputing Centre BDEC for Europe Workshop Barcelona, 28.01.2015 The research leading to these results has received funding from the European

More information

High Throughput WAN Data Transfer with Hadoop-based Storage

High Throughput WAN Data Transfer with Hadoop-based Storage High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San

More information

InfiniBand SDR, DDR, and QDR Technology Guide

InfiniBand SDR, DDR, and QDR Technology Guide White Paper InfiniBand SDR, DDR, and QDR Technology Guide The InfiniBand standard supports single, double, and quadruple data rate that enables an InfiniBand link to transmit more data. This paper discusses

More information

Grid Scheduling Architectures with Globus

Grid Scheduling Architectures with Globus Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents

More information

NAS for Server Virtualization Dennis Chapman Senior Technical Director NetApp

NAS for Server Virtualization Dennis Chapman Senior Technical Director NetApp NAS for Server Virtualization Dennis Chapman Senior Technical Director NetApp Agenda The Landscape has Changed New Customer Requirements The Market has Begun to Move Comparing Performance Results Storage

More information

DISCERN Libraries User Guide

DISCERN Libraries User Guide Distributed Intelligence for Cost-Effective and Reliable Distribution Network Operation DISCERN Libraries User Guide Author: OFFIS Date: 22.04.2016 www.discern.eu The research leading to these results

More information

The Canadian CyberSKA Project

The Canadian CyberSKA Project The Canadian CyberSKA Project A. G. Willis (on behalf of the CyberSKA Project Team) National Research Council of Canada Herzberg Institute of Astrophysics Dominion Radio Astrophysical Observatory May 24,

More information

Deliverable D10.1 Launch and management of dedicated website and social media

Deliverable D10.1 Launch and management of dedicated website and social media STAR-ProBio Sustainability Transition Assessment and Research of Bio-based Products Grant Agreement Number 727740 Deliverable D10.1 Launch and management of dedicated website and social media Version 1.0,

More information

Ref. Ares(2015) /12/2015. D9.1 Project Collaborative Workspace Bénédicte Ferreira, IT

Ref. Ares(2015) /12/2015. D9.1 Project Collaborative Workspace Bénédicte Ferreira, IT Ref. Ares(2015)6013006-22/12/2015 D9.1 Project Collaborative Workspace Bénédicte Ferreira, IT AQUAEXCEL 2020 Deliverable D9.1 Executive Summary Objectives: This deliverable presents the AQUAEXCEL 2020

More information

THE VLBA SENSITIVITY UPGRADE

THE VLBA SENSITIVITY UPGRADE THE VLBA SENSITIVITY UPGRADE Craig Walker, NRAO Eleventh Synthesis Imaging Workshop Socorro, June 10-17, 2008 CONTEXT 2 The VLBA is based on 20 year old technology Only limited new capabilities have been

More information

The challenges of (non-)openness:

The challenges of (non-)openness: The challenges of (non-)openness: Trust and Identity in Research and Education. DEI 2018, Zagreb, April 2018 Ann Harding, SWITCH/GEANT @hardingar Who am I? Why am I here? Medieval History, Computer Science

More information

Grid technologies, solutions and concepts in the synchrotron Elettra

Grid technologies, solutions and concepts in the synchrotron Elettra Grid technologies, solutions and concepts in the synchrotron Elettra Roberto Pugliese, George Kourousias, Alessio Curri, Milan Prica, Andrea Del Linz Scientific Computing Group, Elettra Sincrotrone, Trieste,

More information

MULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis

MULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis MULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis EU H2020 FETHPC project ANTAREX (g.a. 671623) EU FP7 ERC Project MULTITHERMAN (g.a.291125) EETHPC,

More information

The DEEP-ER take on I/O

The DEEP-ER take on I/O Wolfgang Frings Jülich Supercomputing Centre Workshop Exascale I/O: Challenges, Innovations and Solutions SC16, Salt Lake City 18 November 2016 The research leading to these results has received funding

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head The LHC and Detectors Outline Computing Challenges Current

More information

in Horizon 2020 Philippe Froissard European Commission DG Research & Innovation Research Infrastructures

in Horizon 2020 Philippe Froissard European Commission DG Research & Innovation Research Infrastructures Research Infrastructures in Horizon 2020 Philippe Froissard European Commission DG Research & Innovation Research Infrastructures Research infrastructures are facilities, resources, organisational systems

More information

Europeana DSI 2 Access to Digital Resources of European Heritage

Europeana DSI 2 Access to Digital Resources of European Heritage Europeana DSI 2 Access to Digital Resources of European Heritage MILESTONE Revision 1.0 Date of submission 28.04.2017 Author(s) Krystian Adamski, Tarek Alkhaeir, Marcin Heliński, Aleksandra Nowak, Marcin

More information

Advanced Cloud Infrastructures

Advanced Cloud Infrastructures Advanced Cloud Infrastructures From Data Centers to Fog Computing (part 1) Guillaume Pierre Master 2 CCS & SIF, 2017 Advanced Cloud Infrastructures 1 / 35 Advanced Cloud Infrastructures 2 / 35 Advanced

More information

Moving e-infrastructure into a new era the FP7 challenge

Moving e-infrastructure into a new era the FP7 challenge GARR Conference 18 May 2006 Moving e-infrastructure into a new era the FP7 challenge Mário Campolargo European Commission - DG INFSO Head of Unit Research Infrastructures Example of e-science challenges

More information

MULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis

MULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis MULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis EU H2020 FETHPC project ANTAREX (g.a. 671623) EU FP7 ERC Project MULTITHERMAN (g.a.291125) HPC

More information

INNOVER-EAST. WP5 Dissemination. Deliverable D5.3 Basic Dissemination Material

INNOVER-EAST. WP5 Dissemination. Deliverable D5.3 Basic Dissemination Material INNOVER-EAST Building a more effective pathway leading from research to innovation through cooperation between the European Union and Eastern Partnership countries in the field of energy efficiency FP7

More information

Metal Recovery from Low Grade Ores and Wastes Plus

Metal Recovery from Low Grade Ores and Wastes Plus Metal Recovery from Low Grade Ores and Wastes Plus D7.1 Project and public website Public Authors: Marta Macias, Carlos Leyva (IDENER) D7.1 I Page 2 Deliverable Number 7.1 Deliverable Name Project and

More information

Deliverable 6.4. Initial Data Management Plan. RINGO (GA no ) PUBLIC; R. Readiness of ICOS for Necessities of integrated Global Observations

Deliverable 6.4. Initial Data Management Plan. RINGO (GA no ) PUBLIC; R. Readiness of ICOS for Necessities of integrated Global Observations Ref. Ares(2017)3291958-30/06/2017 Readiness of ICOS for Necessities of integrated Global Observations Deliverable 6.4 Initial Data Management Plan RINGO (GA no 730944) PUBLIC; R RINGO D6.5, Initial Risk

More information

Astronomy and Big Data challenge. Sandra Jaque, CTO, REUNA

Astronomy and Big Data challenge. Sandra Jaque, CTO, REUNA Astronomy and Big Data challenge Sandra Jaque, CTO, REUNA Astronomy 70% of the global telescope facilities will be located in Chile by 2020 2021 2025 2022 2017 Astronomy Currently 75% of the astronomy

More information

Euro-BioImaging Preparatory Phase II Project

Euro-BioImaging Preparatory Phase II Project Euro-BioImaging Preparatory Phase II Project Web access portal running with all features, including links to the EuBI nodes' online resources Project N. 688945 Project Title Euro-BioImaging Preparatory

More information

Milestone Solution Partner IT Infrastructure Components Certification Report

Milestone Solution Partner IT Infrastructure Components Certification Report Milestone Solution Partner IT Infrastructure Components Certification Report Huawei OceanStor 5500 V3 SAN Storage 08-09-2015 Table of Contents Executive Summary:... 4 Abstract... 4 Certified Products...

More information

National R&E Networks: Engines for innovation in research

National R&E Networks: Engines for innovation in research National R&E Networks: Engines for innovation in research Erik-Jan Bos EGI Technical Forum 2010 Amsterdam, The Netherlands September 15, 2010 Erik-Jan Bos - Chief Technology Officer at Dutch NREN SURFnet

More information

H2020 EUB EU-Brazil Research and Development Cooperation in Advanced Cyber Infrastructure. NCP Training Brussels, 18 September 2014

H2020 EUB EU-Brazil Research and Development Cooperation in Advanced Cyber Infrastructure. NCP Training Brussels, 18 September 2014 H2020 EUB 2015 EU-Brazil Research and Development Cooperation in Advanced Cyber Infrastructure NCP Training Brussels, 18 September 2014 H2020 EUB 2015 This topic is a major element for the implementation

More information

Helix Nebula The Science Cloud

Helix Nebula The Science Cloud Helix Nebula The Science Cloud CERN 14 May 2014 Bob Jones (CERN) This document produced by Members of the Helix Nebula consortium is licensed under a Creative Commons Attribution 3.0 Unported License.

More information

Cisco Borderless Mobility. Kim Min Se Technical Marketing Engineer Borderless Networks

Cisco Borderless Mobility. Kim Min Se Technical Marketing Engineer Borderless Networks Cisco Borderless Mobility Kim Min Se Technical Marketing Engineer Borderless Networks Mobility as a Business Imperative Done Wrong! Loss of connectivity Inability to attract and retain the best employees

More information

The Virtual Observatory and the IVOA

The Virtual Observatory and the IVOA The Virtual Observatory and the IVOA The Virtual Observatory Emergence of the Virtual Observatory concept by 2000 Concerns about the data avalanche, with in mind in particular very large surveys such as

More information