THE SQUARE KILOMETER ARRAY (SKA) ESD USE CASE
|
|
- Aron Russell
- 6 years ago
- Views:
Transcription
1 THE SQUARE KILOMETER ARRAY (SKA) ESD USE CASE Ronald Nijboer Head ASTRON R&D Computing Group With material from Chris Broekema (ASTRON) John Romein (ASTRON) Nick Rees (SKA Office) Miles Deegan (SKA Office) John Taylor (U. of Cambridge) Michael Wise (ASTRON) 1
2 ASTRON, Offices & Locations Dwingeloo Groningen LOFAR / WSRT Operations LOFAR CEP R&D, Science JIVE, NOVA Borger-Odoorn, Exloo LOFAR core Westerbork WSRT 2
3 Radio Astronomy Doppler shift 21 cm line Galaxy M81 3
4 Square Kilometer Array (SKA) SKA: one Observatory (HQ in UK), two sites (South-Africa & Australia) 4
5 SKA: big scientific questions Testing gravitation Epoch of Reionisation Cosmic Magnetism Cradle of life Large scale structures Turbulent Universe 5
6 SKA Context Diagram The Science Data Processor transforms the Signals into Science Data Products SDP is off-site! (Perth & Cape Town) 6
7 Regional Science Centers Regional Centers are proposed for doing Science with the SKA Data Products 7
8 RSC Functionality Data Discovery Observation database Associated metadata Quick-look data products Flexible catalog queries Integration with VO tools Publish data to VO Data Processing Data Mining Multi-wavelength studies Catalog cross-matching Light-curve analysis Transient classification Feature detection Visualization Reprocessing and calibration High resolution imaging Mosaicing Source extraction Catalog re-creation DM searches 8
9 RSC Requirements Regional Science Centers are being discussed and planned for Requirements do not exist yet H2020 project Aeneas submitted Likely RSCs will be different in different locations SKA SDP type processing will be needed, as well as Data Discovery and Data Mining Design and specification of a distributed, European Science Data Centre (ESDC) to support the pan-european astronomical community in achieving the scientific goals of the SKA 9
10 SDP Key Performance Requirements SDP Local Monitoring & Control C S P Data Processor High Performance ~100 PetaFLOPS Data Intensive ~100 PetaBytes/observation (job) Partially real-time ~10s response time Data Preservation High Volume & High Growth Rate ~100 PetaByte/year Infrequent Access ~few times/year max Delivery System Data Distribution ~100 PetaByte/year from Cape Town & Perth to rest of World Data Discovery Visualisation of 100k by 100k by 100k voxel cubes Partially iterative ~1 Tbytes -1 ~200 ~10 ~10 iterations/job (~6hour) Gbytes -1 Gbytes -1 Observatory 10
11 SDP Functional Breakdown 11
12 Data Parallelism Time & baseline Frequency Visibility data Exploit frequency independence o o o Data parallelism: Dominated by frequency Provides dominant scaling Nothing more needed if each processing node can manage a frequency channel complete processing Grid and degrid Buffered UV data FFT Processing nodes A lot of the processing is embarrasingly (data) parallel, but there will be synchronisation points where data needs to be combined 12
13 SDP Compute Requirements ~50 PFLOPS total sustained, max FFT and Gridding dominant Mixed precision Achieve 10-15% of peak now Large fast working memory (~2 FLOP/byte) Can exchange memory for FLOPs using facetting Fast Storage ~3 Tb/s write, ~30 Tb/s read ~ FLOPS/byte read ~5MW per site 13
14 SDP Compute Characteristics Few, well known applications -> co-design Trivially parallel workloads, baseline architecture leverages this Low arithmatic intensity, thus I/O bound Pseudo real-time + fast storage + batch processing Tight budgets (energy, capital and ops) 14
15 Current Timeline SKA Pre-Construction SKA Construction 2020 Start Early Science 2023 Start Full Operations 15
16 Conclusions SKA is a huge computational challenge RSCs in the process of being defined SDP ~ 50 Pflop (sustained), 5 MW Power is also a major driver. Software complexity is also beyond what has been achieved in astronomy previously. Traditional HPC is not a good match because the problem is bandwidth dominated. SKA would be a perfect Use Case as Big Data application for the EsD projects 16
17 Questions? 17
Data Processing for the Square Kilometre Array Telescope
Data Processing for the Square Kilometre Array Telescope Streaming Workshop Indianapolis th October Bojan Nikolic Astrophysics Group, Cavendish Lab University of Cambridge Email: b.nikolic@mrao.cam.ac.uk
More informationThe Square Kilometre Array. Miles Deegan Project Manager, Science Data Processor & Telescope Manager
The Square Kilometre Array Miles Deegan Project Manager, Science Data Processor & Telescope Manager The Square Kilometre Array (SKA) The SKA is a next-generation radio interferometer: 3 telescopes, on
More informationNetherlands Institute for Radio Astronomy. May 18th, 2009 Hanno Holties
Netherlands Institute for Radio Astronomy Update LOFAR Long Term Archive May 18th, 2009 Hanno Holties LOFAR Long Term Archive (LTA) Update Status Architecture Data Management Integration LOFAR, Target,
More informationJohn W. Romein. Netherlands Institute for Radio Astronomy (ASTRON) Dwingeloo, the Netherlands
Signal Processing on GPUs for Radio Telescopes John W. Romein Netherlands Institute for Radio Astronomy (ASTRON) Dwingeloo, the Netherlands 1 Overview radio telescopes six radio telescope algorithms on
More informationSKA Computing and Software
SKA Computing and Software Nick Rees 18 May 2016 Summary Introduc)on System overview Compu)ng Elements of the SKA Telescope Manager Low Frequency Aperture Array Central Signal Processor Science Data Processor
More informationPARALLEL PROGRAMMING MANY-CORE COMPUTING: THE LOFAR SOFTWARE TELESCOPE (5/5)
PARALLEL PROGRAMMING MANY-CORE COMPUTING: THE LOFAR SOFTWARE TELESCOPE (5/5) Rob van Nieuwpoort Vrije Universiteit Amsterdam & Astron, the Netherlands Institute for Radio Astronomy Why Radio? Credit: NASA/IPAC
More informationEnergy-Efficient Data Transfers in Radio Astronomy with Software UDP RDMA Third Workshop on Innovating the Network for Data-Intensive Science, INDIS16
Energy-Efficient Data Transfers in Radio Astronomy with Software UDP RDMA Third Workshop on Innovating the Network for Data-Intensive Science, INDIS16 Przemek Lenkiewicz, Researcher@IBM Netherlands Bernard
More informationThe challenges of computing at astronomical scale
Netherlands Institute for Radio Astronomy The challenges of computing at astronomical scale Chris Broekema Thursday 15th February, 2018, New Zealand SKA Forum 2018, Auckland, New Zealand ASTRON is part
More informationComputational issues for HI
Computational issues for HI Tim Cornwell, Square Kilometre Array How SKA processes data Science Data Processing system is part of the telescope Only one system per telescope Data flow so large that dedicated
More informationASKAP Central Processor: Design and Implementa8on
ASKAP Central Processor: Design and Implementa8on Calibra8on and Imaging Workshop 2014 Ben Humphreys ASKAP So(ware and Compu3ng Project Engineer 3rd - 7th March 2014 ASTRONOMY AND SPACE SCIENCE Australian
More informationPARALLEL PROGRAMMING MANY-CORE COMPUTING FOR THE LOFAR TELESCOPE ROB VAN NIEUWPOORT. Rob van Nieuwpoort
PARALLEL PROGRAMMING MANY-CORE COMPUTING FOR THE LOFAR TELESCOPE ROB VAN NIEUWPOORT Rob van Nieuwpoort rob@cs.vu.nl Who am I 10 years of Grid / Cloud computing 6 years of many-core computing, radio astronomy
More informationGPUS FOR NGVLA. M Clark, April 2015
S FOR NGVLA M Clark, April 2015 GAMING DESIGN ENTERPRISE VIRTUALIZATION HPC & CLOUD SERVICE PROVIDERS AUTONOMOUS MACHINES PC DATA CENTER MOBILE The World Leader in Visual Computing 2 What is a? Tesla K40
More informationNew Zealand Involvement in Solving the SKA Computing Challenges
New Zealand Involvement in Solving the SKA Computing Challenges D R ANDREW E N S O R D I R ECTO R H P C R ESEARC H L A B O R ATORY/ D I R ECTOR N Z SKA ALLIANCE COMPUTING FO R S K A COLLO Q U I UM 2 0
More informationEuropean VLBI Network
European VLBI Network Cormac Reynolds, JIVE European Radio Interferometry School, Bonn 12 Sept. 2007 EVN Array 15 dissimilar telescopes Observes 3 times a year (approx 60 days per year) Includes some
More informationPDR.01.01: ASSUMPTIONS AND NON-CONFORMANCE
PDR.0.0: ASSUMPTIONS AND NON-CONFORMANCE Document number Context MGT Revision Authors Ian Cooper, Ronald Nijboer Release Date 205-02-09 Document Classification Status Draft 205-02-09 Page of 6 Name Designation
More informationSDP Design for Cloudy Regions
SDP Design for Cloudy Regions Markus Dolensky, 11/02/2016 2 ICRAR s Data Intensive Astronomy Group M.B. I.C. R.D. M.D. K.V. C.W. A.W. D.P. R.T. generously borrowed content from above colleagues 3 SDP Subelements
More informationTHE EUCLID ARCHIVE SYSTEM: A DATA-CENTRIC APPROACH TO BIG DATA
THE EUCLID ARCHIVE SYSTEM: A DATA-CENTRIC APPROACH TO BIG DATA Rees Williams on behalf of A.N.Belikov, D.Boxhoorn, B. Dröge, J.McFarland, A.Tsyganov, E.A. Valentijn University of Groningen, Groningen,
More informationThe Virtual Observatory and the IVOA
The Virtual Observatory and the IVOA The Virtual Observatory Emergence of the Virtual Observatory concept by 2000 Concerns about the data avalanche, with in mind in particular very large surveys such as
More informationSKA SDP : A snapshot of recent technical directions and conclusions
SKA SDP : A snapshot of recent technical directions and conclusions Chris Broekema ASTRON Netherlands Institute for Radio Astronomy Highlights A whirlwind overview of recent prototyping and design work
More informationThe Virtual Observatory in Australia Connecting to International Initiatives. Peter Lamb. CSIRO Mathematical & Information Sciences
The Virtual Observatory in Australia Connecting to International Initiatives Peter Lamb CSIRO Mathematical & Information Sciences The Grid & escience Convergence of high-performance computing, huge data
More informationOverview of Science Data Processor. Paul Alexander Rosie Bolton Ian Cooper Bojan Nikolic Simon Ratcliffe Harry Smith
Overview of Science Data Processor Paul Alexander Rosie Bolton Ian Cooper Bojan Nikolic Simon Ratcliffe Harry Smith Consortium Management and System Engineering Aims of the SDP Work Detailed design of
More informationSKA Regional Centre Activities in Australasia
SKA Regional Centre Activities in Australasia Dr Slava Kitaeff CSIRO-ICRAR APSRC Project Engineer ERIDANUS National Project Lead Why SKA Regional Centres? SKA 1 Observatory Compute capacity: 100 Pflops
More informationSKA SDP-COMP Middleware: The intersect with commodity computing. Piers Harding // February, 2017
SKA SDP-COMP Middleware: The intersect with commodity computing Piers Harding // February, 2017 Overview SDP Middleware why is this important What are the options Middleware where is industry heading What
More informationSquare Kilometre Array
Square Kilometre Array C4SKA 16 February 2018 David Luchetti Australian SKA Project Director Presentation 1. Video of the Murchison Radioastronomy Observatory 2. Is SKA a Collaborative project? 3. Australia/New
More informationWhere is the Slack in the SDP System? Markus Dolensky
Where is the Slack in the SDP System? Markus Dolensky SKA Signal & Data Path along Design Data Intensive Astronomy LFAA DISH Correlator CSP Science Data Processor 2 SDP Design Challenge revisited Power
More informationGiovanni Lamanna LAPP - Laboratoire d'annecy-le-vieux de Physique des Particules, Université de Savoie, CNRS/IN2P3, Annecy-le-Vieux, France
Giovanni Lamanna LAPP - Laboratoire d'annecy-le-vieux de Physique des Particules, Université de Savoie, CNRS/IN2P3, Annecy-le-Vieux, France ERF, Big data & Open data Brussels, 7-8 May 2014 EU-T0, Data
More informationThe IPAC Research Archives. Steve Groom IPAC / Caltech
The IPAC Research Archives Steve Groom IPAC / Caltech IPAC overview The Infrared Processing and Analysis Center (IPAC) at Caltech is dedicated to science operations, data archives, and community support
More informationFocus Session on Multi-dimensional Data
Focus Session on Multi-dimensional Data Introduction Mark Allen, Joe Lazio IVOA Interoperability Meeting, ESAC, Madrid, May 20, 2014 CoSADIE Project Science Priority Areas Multi-dimensional Data image:
More informationASKAP Data Flow ASKAP & MWA Archives Meeting
ASKAP Data Flow ASKAP & MWA Archives Meeting Ben Humphreys ASKAP Software and Computing Project Engineer 25 th March 2013 ASTRONOMY AND SPACE SCIENCE ASKAP @ Murchison Radioastronomy Observatory Australian
More informationDesign and Implementation of the Japanese Virtual Observatory (JVO) system Yuji SHIRASAKI National Astronomical Observatory of Japan
Design and Implementation of the Japanese Virtual Observatory (JVO) system Yuji SHIRASAKI National Astronomical Observatory of Japan 1 Introduction What can you do on Japanese Virtual Observatory (JVO)?
More informationThe Programmable Telescope
The Programmable Telescope Tim Cornwell ASKAP Computing Lead New generation of radio telescopes Flourishing around the world Atacama Large Millimetre Array (ALMA) Expanded Very Large Array (EVLA) Square
More informationAdaptive selfcalibration for Allen Telescope Array imaging
Adaptive selfcalibration for Allen Telescope Array imaging Garrett Keating, William C. Barott & Melvyn Wright Radio Astronomy laboratory, University of California, Berkeley, CA, 94720 ABSTRACT Planned
More informationCSD3 The Cambridge Service for Data Driven Discovery. A New National HPC Service for Data Intensive science
CSD3 The Cambridge Service for Data Driven Discovery A New National HPC Service for Data Intensive science Dr Paul Calleja Director of Research Computing University of Cambridge Problem statement Today
More informationData federations and its applications
Data federations and its applications Edwin A. Valentijn Prof Astronomical Information Technology Target- OmegaCEN Kapteyn 20 Oktober 2016 IAU Symp. Astro Informatics Sorrento Astro-WISE Data federations
More informationVisualization & the CASA Viewer
Visualization & the Viewer Juergen Ott & the team Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Visualization Goals:
More informationVIRTUAL OBSERVATORY TECHNOLOGIES
VIRTUAL OBSERVATORY TECHNOLOGIES / The Johns Hopkins University Moore s Law, Big Data! 2 Outline 3 SQL for Big Data Computing where the bytes are Database and GPU integration CUDA from SQL Data intensive
More informationEarly Evaluation of the "Infinite Memory Engine" Burst Buffer Solution
Early Evaluation of the "Infinite Memory Engine" Burst Buffer Solution Wolfram Schenck Faculty of Engineering and Mathematics, Bielefeld University of Applied Sciences, Bielefeld, Germany Salem El Sayed,
More informationPoS(10th EVN Symposium)098
1 Joint Institute for VLBI in Europe P.O. Box 2, 7990 AA Dwingeloo, The Netherlands E-mail: szomoru@jive.nl The, a Joint Research Activity in the RadioNet FP7 2 programme, has as its aim the creation of
More informationUniversity of Groningen
University of Groningen The Euclid Archive System: A Datacentric Approach to Big Data Belikov, Andrey; Williams, Owen; Altieri, Bruno; Boxhoorn, Danny; Buenadicha, Guillermo ; Droge, Bob; McFarland, John;
More informationPDR.01 SDP Architecture
PDR.01 SDP Architecture Document number.. SKA-TEL-SDP-0000013 Context.. ARCH Revision...2 Author. Nikolic Release Date 2015-02-09 Document Classification Status. Draft Release Date: 2015-02-09 Page 1 of
More informationSKA. data processing, storage, distribution. Jean-Marc Denis Head of Strategy, BigData & HPC. Oct. 16th, 2017 Paris. Coyright Atos 2017
SKA data processing, storage, distribution Oct. 16th, 2017 Paris Jean-Marc Denis Head of Strategy, BigData & HPC Coyright Atos 2017 What are SKA data and compute needs? Some key numbers Source: Image Swinburn
More informationData Centres in the Virtual Observatory Age
Data Centres in the Virtual Observatory Age David Schade Canadian Astronomy Data Centre A few things I ve learned in the past two days There exist serious efforts at Long-Term Data Preservation Alliance
More informationMinutes of Meeting LOFAR Software
Minutes of Meeting LOFAR Software Date: 2007-11-28 Next meeting: 2007-12-05 9:15-10:15 Minnaert room Present: Andre Gunst No Ronald Nijboer Ruud Overeem John Romein Michael Wise cc: Arthur Coolen, Jurjen
More informationImage Pixelization and Dynamic Range
EVLA Memo 114 Image Pixelization and Dynamic Range W. D. Cotton, Juan M. Uson NRAO 1 Abstract This study investigates some of the effects of representing the sky by a rectangular grid of pixels on the
More informationData Mining, Parallelism, Data Mining, Parallelism, and Grids. Queen s University, Kingston David Skillicorn
Data Mining, Parallelism, Data Mining, Parallelism, and Grids David Skillicorn Queen s University, Kingston skill@cs.queensu.ca Data mining builds models from data in the hope that these models reveal
More informationBuilding on Existing Communities: the Virtual Astronomical Observatory (and NIST)
Building on Existing Communities: the Virtual Astronomical Observatory (and NIST) Robert Hanisch Space Telescope Science Institute Director, Virtual Astronomical Observatory Data in astronomy 2 ~70 major
More informationCyberSKA: Project Update. Cameron Kiddle, CyberSKA Technical Coordinator
CyberSKA: Project Update Cameron Kiddle, CyberSKA Technical Coordinator What is CyberSKA? Initiative to develop a scalable and distributed cyberinfrastructure platform to meet evolving science needs of
More informationCase Study: CyberSKA - A Collaborative Platform for Data Intensive Radio Astronomy
Case Study: CyberSKA - A Collaborative Platform for Data Intensive Radio Astronomy Outline Motivation / Overview Participants / Industry Partners Documentation Architecture Current Status and Services
More informationRUSSIAN DATA INTENSIVE GRID (RDIG): CURRENT STATUS AND PERSPECTIVES TOWARD NATIONAL GRID INITIATIVE
RUSSIAN DATA INTENSIVE GRID (RDIG): CURRENT STATUS AND PERSPECTIVES TOWARD NATIONAL GRID INITIATIVE Viacheslav Ilyin Alexander Kryukov Vladimir Korenkov Yuri Ryabov Aleksey Soldatov (SINP, MSU), (SINP,
More informationExploiting Virtual Observatory and Information Technology: Techniques for Astronomy
Exploiting Virtual Observatory and Information Technology: Techniques for Astronomy Nicholas Walton AstroGrid Project Scientist Institute of Astronomy, The University of Cambridge Lecture #3 Goal: Applications
More informationSKA Technical developments relevant to the National Facility. Keith Grainge University of Manchester
SKA Technical developments relevant to the National Facility Keith Grainge University of Manchester Talk Overview SKA overview Receptors Data transport and network management Synchronisation and timing
More informationPoS(EXPReS09)023. e-evn progress. 3 years of EXPReS. Arpad Szomoru 1
3 years of EXPReS 1 Joint Institute for VLBI in Europe P.O. Box 2, 7990 AA Dwingeloo, The Netherlands E-mail: szomoru@jive.nl During the past years e-vlbi has undergone a remarkable development. At the
More informationTHE EUCLID ARCHIVE SYSTEM: A DATA-CENTRIC APPROACH TO BIG DATA
THE EUCLID ARCHIVE SYSTEM: A DATA-CENTRIC APPROACH TO BIG DATA Sara Nieto on behalf of B.Altieri, G.Buenadicha, J. Salgado, P. de Teodoro European Space Astronomy Center, European Space Agency, Spain O.R.
More informationPART I - Fundamentals of Parallel Computing
PART I - Fundamentals of Parallel Computing Objectives What is scientific computing? The need for more computing power The need for parallel computing and parallel programs 1 What is scientific computing?
More informationSDP Memo 048: Two Dimensional Sparse Fourier Transform Algorithms
SDP Memo 048: Two Dimensional Sparse Fourier Transform Algorithms Document Number......................................................... SDP Memo 048 Document Type.....................................................................
More informationUK LUG 10 th July Lustre at Exascale. Eric Barton. CTO Whamcloud, Inc Whamcloud, Inc.
UK LUG 10 th July 2012 Lustre at Exascale Eric Barton CTO Whamcloud, Inc. eeb@whamcloud.com Agenda Exascale I/O requirements Exascale I/O model 3 Lustre at Exascale - UK LUG 10th July 2012 Exascale I/O
More informationFast Holographic Deconvolution
Precision image-domain deconvolution for radio astronomy Ian Sullivan University of Washington 4/19/2013 Precision imaging Modern imaging algorithms grid visibility data using sophisticated beam models
More informationLessons learnt from implementing mosaicing and faceting in ASKAPsoft. Max Voronkov & Tim Cornwell ASKAP team 2nd April 2009
Lessons learnt from implementing mosaicing and faceting in ASKAPsoft Max Voronkov & Tim Cornwell ASKAP team 2nd April 2009 Outline - Imaging software ASKAPsoft re-uses LOFAR design Imaging is treated as
More informationEVLA Memo #132 Report on the findings of the CASA Terabyte Initiative: Single-node tests
EVLA Memo #132 Report on the findings of the CASA Terabyte Initiative: Single-node tests S. Bhatnagar NRAO, Socorro May 18, 2009 Abstract This note reports on the findings of the Terabyte-Initiative of
More informationSupport for Large Scale HPC and Big Data Science Projects in South Africa: SKA and More. 25 th September 2017
Support for Large Scale HPC and Big Data Science Projects in South Africa: SKA and More 25 th September 2017 Background: The CHPC in SA CHPC User Community: South Africa CHPC Existing Users Future Users
More informationMoving e-infrastructure into a new era the FP7 challenge
GARR Conference 18 May 2006 Moving e-infrastructure into a new era the FP7 challenge Mário Campolargo European Commission - DG INFSO Head of Unit Research Infrastructures Example of e-science challenges
More informationImaging and Deconvolution
Imaging and Deconvolution Urvashi Rau National Radio Astronomy Observatory, Socorro, NM, USA The van-cittert Zernike theorem Ei E V ij u, v = I l, m e sky j 2 i ul vm dldm 2D Fourier transform : Image
More informationIntel PSG (Altera) Enabling the SKA Community. Lance Brown Sr. Strategic & Technical Marketing Mgr.
Intel PSG (Altera) Enabling the SKA Community Lance Brown Sr. Strategic & Technical Marketing Mgr. lbrown@altera.com, 719-291-7280 Agenda Intel Programmable Solutions Group (Altera) PSG s COTS Strategy
More informationDSH GUI. A. Marassi for the DSH.LMC team. Tango LMC Harmonization Meeting Madrid Apr 2016
DSH GUI A. Marassi for the DSH.LMC team Tango LMC Harmonization Meeting Madrid - 11-13 Apr 2016 Outline DSH GUI SKA elements LMC GUIs DSH.LMC GUI specs LOFAR GUIs LOFAR Navigator GUI DSH.LMC GUIs 2 SKA
More informationIVOA Cape Town Interop Closing Remarks
IVOA Cape Town Interop Closing Remarks Christophe Arviset Chair of the IVOA Executive Committee IVOA Interop Meeting Closing Session Cape Town, South Africa, 13/May/2016 Vision of the IVOA Develop an interoperable
More informationNational Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology WISE Archive.
Bruce Berriman / Steve Groom Infrared Science Archive (IRSA), IPAC/Caltech GBB/SLG - 1 WSDC Functional Block Diagram White Sands JPL UCLA HRP H/K MOS Maneuvers SOC Science Team Images FTP Site Ancillary
More informationHigh Performance Computing. What is it used for and why?
High Performance Computing What is it used for and why? Overview What is it used for? Drivers for HPC Examples of usage Why do you need to learn the basics? Hardware layout and structure matters Serial
More informationLessons from EXPReS and the future of e-vlbi. 2nd Terena E2E Workshop, Amsterdam,
Lessons from EXPReS and the future of e-vlbi 2nd Terena E2E Workshop, Amsterdam, 2009-12-08 What is JIVE? Operate the EVN correlator and support astronomers doing VLBI. A collaboration of the major radioastronomical
More informationSome Reflections on Advanced Geocomputations and the Data Deluge
Some Reflections on Advanced Geocomputations and the Data Deluge J. A. Rod Blais Dept. of Geomatics Engineering Pacific Institute for the Mathematical Sciences University of Calgary, Calgary, AB www.ucalgary.ca/~blais
More information( ) = First Bessel function, x = π Dθ
Observational Astronomy Image formation Complex Pupil Function (CPF): (3.3.1) CPF = P( r,ϕ )e ( ) ikw r,ϕ P( r,ϕ ) = Transmittance of the aperture (unobscured P = 1, obscured P = 0 ) k = π λ = Wave number
More informationProgress towards an Asia-Pacific Regional Centre Network. Peter Quinn (ICRAR)
Progress towards an Asia-Pacific Regional Centre Network Peter Quinn (ICRAR) What observatories do Regional Centre? Regional Centre Regional Centre? 2 Data rates and volumes ALMA Science Data Products
More informationDistributed Archive System for the Cherenkov Telescope Array
Distributed Archive System for the Cherenkov Telescope Array RIA-653549 Eva Sciacca, S. Gallozzi, A. Antonelli, A. Costa INAF, Astrophysical Observatory of Catania INAF, Astronomical Observatory of Rome
More informationProgrammable Information Highway (with no Traffic Jams)
Programmable Information Highway (with no Traffic Jams) Inder Monga Energy Sciences Network Scientific Networking Division Lawrence Berkeley National Lab Exponential Growth ESnet Accepted Traffic: Jan
More informationHigh Performance Computing from an EU perspective
High Performance Computing from an EU perspective DEISA PRACE Symposium 2010 Barcelona, 10 May 2010 Kostas Glinos European Commission - DG INFSO Head of Unit GÉANT & e-infrastructures 1 "The views expressed
More informationAuto-Tuning Dedispersion for Many-Core Accelerators
Auto-Tuning Dedispersion for Many-Core Accelerators Alessio Sclocco, Henri E. Bal Faculty of Sciences Vrije Universiteit Amsterdam Amsterdam, the Netherlands a.sclocco@vu.nl, h.e.bal@vu.nl Jason Hessels,
More informationTHE SKA SCIENCE AND TECHNICAL OPERATIONS PLAN
THE SKA SCIENCE AND TECHNICAL OPERATIONS PLAN Document number...wp2-001.010.010-pla-001 Revision... A Author... SKA Operations Working Group Date... 2010-01-29 Status... Approved for release Name Designation
More informationExploiting the OpenPOWER Platform for Big Data Analytics and Cognitive. Rajesh Bordawekar and Ruchir Puri IBM T. J. Watson Research Center
Exploiting the OpenPOWER Platform for Big Data Analytics and Cognitive Rajesh Bordawekar and Ruchir Puri IBM T. J. Watson Research Center 3/17/2015 2014 IBM Corporation Outline IBM OpenPower Platform Accelerating
More informationTypically applied in clusters and grids Loosely-coupled applications with sequential jobs Large amounts of computing for long periods of times
Typically applied in clusters and grids Loosely-coupled applications with sequential jobs Large amounts of computing for long periods of times Measured in operations per month or years 2 Bridge the gap
More informationHECToR. UK National Supercomputing Service. Andy Turner & Chris Johnson
HECToR UK National Supercomputing Service Andy Turner & Chris Johnson Outline EPCC HECToR Introduction HECToR Phase 3 Introduction to AMD Bulldozer Architecture Performance Application placement the hardware
More informationSDP Science Pipeline Management Use Case View
SDP Science Pipeline Management Use Case View M. Ashdown TABLE OF CONTENTS 1. Primary Representation 2 2. Element Catalogue 2 2.1. Elements and Their Properties 2 2.1.1 Science Operations 2 2.1.3 Manage
More informationHigh Performance Computing. What is it used for and why?
High Performance Computing What is it used for and why? Overview What is it used for? Drivers for HPC Examples of usage Why do you need to learn the basics? Hardware layout and structure matters Serial
More informationWorkshop: Innovation Procurement in Horizon 2020 PCP Contractors wanted
Workshop: Innovation Procurement in Horizon 2020 PCP Contractors wanted Supercomputing Centre Institute for Advanced Simulation / FZJ 1 www.prace-ri.eu Challenges: Aging Society Energy Food How we can
More informationTheme 7 Group 2 Data mining technologies Catalogues crossmatching on distributed database and application on MWA absorption source finding
Theme 7 Group 2 Data mining technologies Catalogues crossmatching on distributed database and application on MWA absorption source finding Crossmatching is a method to find corresponding objects in different
More informationCASA. Algorithms R&D. S. Bhatnagar. NRAO, Socorro
Algorithms R&D S. Bhatnagar NRAO, Socorro Outline Broad areas of work 1. Processing for wide-field wide-band imaging Full-beam, Mosaic, wide-band, full-polarization Wide-band continuum and spectral-line
More informationAdvanced Multi-Beam Spect rom et er for t he GBT
Advanced Multi-Beam Spect rom et er for t he Conceptual Software Design Amy Shelton, Patrick Brandt, Bob Garwood, Joe Masters, Mark Whitehead NRAO and U.C. Berkeley Joint Conceptual Design Review January
More informationChallenges and Opportunities with Big Data. By: Rohit Ranjan
Challenges and Opportunities with Big Data By: Rohit Ranjan Introduction What is Big Data? Big data is data sets that are so voluminous and complex that traditional data processing application software
More informationI VLB A Correlator Memo No.jkfLl (860506)
I VLB A Correlator Memo No.jkfLl (860506) National Radio Astronomy Observatory Charlottesville, Virginia 5 May, 1986 To: VLBA Correlator Memo Series From: John Benson, Ray Escoffier Subject: On-line Computers
More informationThe Hopper System: How the Largest* XE6 in the World Went From Requirements to Reality! Katie Antypas, Tina Butler, and Jonathan Carter
The Hopper System: How the Largest* XE6 in the World Went From Requirements to Reality! Katie Antypas, Tina Butler, and Jonathan Carter CUG 2011, May 25th, 2011 1 Requirements to Reality Develop RFP Select
More informationEGI: Linking digital resources across Eastern Europe for European science and innovation
EGI- InSPIRE EGI: Linking digital resources across Eastern Europe for European science and innovation Steven Newhouse EGI.eu Director 12/19/12 EPE 2012 1 EGI European Over 35 countries Grid Secure sharing
More informationD Demonstration of integrated BoD testing and validation
D6.06 - Demonstration of integrated BoD testing and validation Deliverable Number 06 WP # 6 Date 2012-12-30 Authors Jimmy Cullen, The University of Manchester Fedde Bloemhof, JIVE Paul Boven, JIVE Ralph
More informationSynonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface (MPI) Large of amounts of computing for short
Synonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface (MPI) Large of amounts of computing for short periods of time Usually requires low latency interconnects
More informationParametrising spatially resolved HI disks
Netherlands Institute for Radio Astronomy Parametrising spatially resolved HI disks Gyula I. G. Józsa ASTRON is part of the Netherlands Organisation for Scientific Research (NWO) Rotation curves CDM makes
More informationWorkload Management. Stefano Lacaprara. CMS Physics Week, FNAL, 12/16 April Department of Physics INFN and University of Padova
Workload Management Stefano Lacaprara Department of Physics INFN and University of Padova CMS Physics Week, FNAL, 12/16 April 2005 Outline 1 Workload Management: the CMS way General Architecture Present
More informationNFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC
Segregated storage and compute NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Co-located storage and compute HDFS, GFS Data
More informationAMAZON S3 FOR SCIENCE GRIDS: A VIABLE SOLUTION?
AMAZON S3 FOR SCIENCE GRIDS: A VIABLE SOLUTION? Mayur Palankar and Adriana Iamnitchi University of South Florida Matei Ripeanu University of British Columbia Simson Garfinkel Harvard University Amazon
More informationExercise Architecture of Parallel Computer Systems
Exercise Architecture of Parallel Computer Systems SoSe 18 L.079.05810 www.uni-paderborn.de/pc2 Architecture of Parallel Computer Systems SoSe 18 J.Simon 1 Overview Computer Systems Test Cluster (Arminius)
More informationAccessing and Exploiting Solar Dynamics Observatory (SDO) Data in Europe. Veronique Delouille
Accessing and Exploiting Solar Dynamics Observatory (SDO) Data in Europe Veronique Delouille SWWT plenary meeting, 28 June 2011 Introduction As from NASA policy: Data from SDO mission are freely available
More informationDesigning the Future Data Management Environment for [Radio] Astronomy. JJ Kavelaars Canadian Astronomy Data Centre
Designing the Future Data Management Environment for [Radio] Astronomy JJ Kavelaars Canadian Astronomy Data Centre 2 Started working in Radio Data Archiving as Graduate student at Queen s in 1993 Canadian
More informationThe Computation and Data Needs of Canadian Astronomy
Summary The Computation and Data Needs of Canadian Astronomy The Computation and Data Committee In this white paper, we review the role of computing in astronomy and astrophysics and present the Computation
More informationData Processing for SUBARU Telescope using GRID
Data Processing for SUBARU Telescope using GRID Y. Shirasaki M. Tanaka S. Kawanomoto S. Honda M. Ohishi Y. Mizumoto National Astronomical Observatory of Japan, 2-21-1, Osawa, Mitaka, Tokyo 181-8588, Japan
More information