Uniform Resource Locator Wide Area Network World Climate Research Programme Coupled Model Intercomparison

Size: px
Start display at page:

Download "Uniform Resource Locator Wide Area Network World Climate Research Programme Coupled Model Intercomparison"

Transcription

1 Glossary API Application Programming Interface AR5 IPCC Assessment Report 4 ASCII American Standard Code for Information Interchange BUFR Binary Universal Form for the Representation of meteorological data C3Grid Collaborative Climate Community Data and Processing Grid CDAT Climate Data Analysis Tools CDM Common Data Model CDO Climate Data Operator CF Climate and Forecast CMOR Climate Model Output Rewriter DAP Data Access Protocol DAPPER OPeNDAP Server for in-situ and gridded data DAS Data Attribute Structure DDS Dataset Descriptor Structure DDX Dataset Descriptor Structure (DDS) in Extensible Markup Language (XML) format DIF Directory Interchange Format ECMWF European Centre for Medium-Range Weather Forecasts ESG Earth System Grid ESM Earth System Model ESMF Earth System Modeling Framework ESnet Energy Sciences Network FFIO Flexible Formatted I/O FGDC Federal Geographic Data Committee FMS Flexible Modeling System FTP File Transfer Protocol GCDM Global Change Master Directory GCM General Circulation Model GDAL Geospatial Data Abstraction Library GDS Grid Analysis and Display System (GrADS)-Data Server GIS Geographical Information System GMAO Global Modeling and Assimilation Office V. Balaji et al., Earth System Modelling Volume 4, SpringerBriefs in Earth System Sciences, DOI: / , Ó The Author(s)

2 54 Glossary GPFS General Parallel File System GrADS Grid Analysis and Display System GRIB GRIdded Binary GridFTP Grid File Transfer Protocol HDF Hierarchical Data Format HDF EOS Hierarchical Data Format Earth Observing System HPC High Performance Computing HPSS High Performance Storage System HSM Hierarchical Storage Management HTTP Hypertext Transfer Protocol IDL Interactive Data Language IEEE Institute of Electrical and Electronics Engineers IPCC Intergovernmental Panel on Climate Change JPEG Joint Photographic Experts Group MMD Multi-model Database MPI Message Passing Interface NASA National Aeronautics and Space Administration NCEP National Centers for Environmental Prediction NCL National Center for Atmospheric Research (NCAR) Command Language NcML NetCDF Markup Language NCO NetCDF Operator NetCDF Network Common Data Form NEXRAD Next-Generation Radar NOAA National Oceanic and Atmospheric Administration NWP Numerical weather prediction OGC Open Geospatial Consortium OPeNDAP Open-source Project for a Network Data Access Protocol PFS Parallel File System PNG Portable Network Graphics POSIX Portable Operating System Interface PP format Post Processing Format, a proprietary file format for meteorological data developed by the UK Met Office PRISM Partnership for Research Infrastructure in Earth System Modeling PVFS Parallel Virtual File System PyDAP Python library implementing the Data Access Protocol RDBMS relational database management system REST Representational State Transfer SAM QFS Storage and Archive-Manager Quick File System SCRIP Spherical Coordinate Remapping and Interpolation Package SSM/I Special Sensor Microwave Imager TDS Thematic Realtime Environmental Distributed Data Services (THREDDS) Data Server THREDDS Thematic Realtime Environmental Distributed Data Services

3 Glossary 55 URL WAN WCRP/CMIP3 WCS WFS WMO WMS XML Uniform Resource Locator Wide Area Network World Climate Research Programme Coupled Model Intercomparison Project phase 3 Web Coverage Service Web Feature Service World Meteorological Organization Web Map Service Extensible Markup Language

4 Index A ADIOS, 15, 18 Aggregation, 18, 19, 34, 35, 37, 42 Asynchronous I/O, 11, 17 B Bulk access, 29 C C3Grid, 44 CDAT, 28, 34, 35, 44, 48 CDM, 33 CDO, 19 CF, 17 CF conventions, 15, 31 CF-IO, 15, 17 CF NetCDF, 9 CMOR, 19 Collective I/O, 8 Common data model, 33 CXFS, 5, 21 D DAPPER, 29 Data-format libraries, 8 Data shipping, 7 Data sieving, 7 Discovery metadata, 32 Distributed field, 14 Distributed grid, 14 Distributed I/O, 15, 16 DMF, E ESG, 44 ESMF, 15, 17 F False sharing, 6, 49 Ferret, 44, 48 File view, 7 FMS, 15, 16 G Georeferencing coordinate systems, 31 GIS, 30, 34 GPFS, 6 GrADS, 28, 29, 44 GrADS-Data Server (GDS), 29 GRIB, 25, 28, 32, 33, 35 H HDF, 26, 32 HDF4, 25, 35 HDF5, 9, 25, 27, 29, 33 HDF-EOS, 25, 27, 32 Hierarchical storage management, 21, 22, 24 High-level I/O libraries, 8 HPSS, HSM, 21, 22, 24, 49 I I/O software stack, 3 IDL, 26, 44 V. Balaji et al., Earth System Modelling Volume 4, SpringerBriefs in Earth System Sciences, DOI: / , Ó The Author(s)

5 58 Index L Linear media, 2, 22, 49 Live access server, 29 Lustre, 6, 22 M MATLAB, 26, 43, 44 MPI datatypes, 7 MPI hints, 8 MPI-IO, 7 mpp_io, 14, 15, 17 Multi-threaded I/O, 15 N NCL, 44, 48 NCO, 19, 44 Ncview, 44 NetCDF, 2, 9, 10, 14, 15, 18, 19, 25 27, 29 33, 35, 37, 42 45, 50 Noncontiguous I/O, 7, 10 Q Quilted I/O, 15 R Read-modify-write, 6 8 Remote, 28 Remote data access, 28 S SAM-QFS, 21 SCRIP, 19, 37, 42 Single-threaded I/O, 15 Storage-class memory, 50 T TDS, 29, 30, 34, 35 Three-level storage model, 22, 23 THREDDS data server, 29 Two-phase I/O, 7, 8 O Object-based storage, 6 OPeNDAP, 29, 30, 33, 37 P PanFS, 6 Parallel file system, 5, 6, 11 Parallel NetCDF, 9 Persistent temporary storage, 23 Post-processing, 2, 18 PRISM, 15, 16 PVFS, 6 PyDAP, 29 U Use metadata, 32 W WCS, 30 WFS, 30 WMS, 30 WRF-IO, 15, 17

Distributed Online Data Access and Analysis

Distributed Online Data Access and Analysis Distributed Online Data Access and Analysis Ruixin Yang George Mason University Slides from SIESIP Partners and from NOMADS PI, Glenn K. Rutledge of US NCDC on NOMADS SIESIP: Seasonal-to-Interannual Earth

More information

HDF Product Designer: A tool for building HDF5 containers with granule metadata

HDF Product Designer: A tool for building HDF5 containers with granule metadata The HDF Group HDF Product Designer: A tool for building HDF5 containers with granule metadata Lindsay Powers Aleksandar Jelenak, Joe Lee, Ted Habermann The HDF Group Data Producer s Conundrum 2 HDF Features

More information

Ocean, Atmosphere & Climate Model Assessment for Everyone

Ocean, Atmosphere & Climate Model Assessment for Everyone Ocean, Atmosphere & Climate Model Assessment for Everyone Rich Signell USGS Woods Hole, MA Unidata 2014 DeSouza Award Presentation Boulder, CO : Sep 15, 2014 2 US Integrated Ocean Observing System (IOOS

More information

Making data access easier with OPeNDAP. James Gallapher (OPeNDAP TM ) Duan Beckett (BoM) Kate Snow (NCI) Robert Davy (CSIRO) Adrian Burton (ARDC)

Making data access easier with OPeNDAP. James Gallapher (OPeNDAP TM ) Duan Beckett (BoM) Kate Snow (NCI) Robert Davy (CSIRO) Adrian Burton (ARDC) Making data access easier with OPeNDAP James Gallapher (OPeNDAP TM ) Duan Beckett (BoM) Kate Snow (NCI) Robert Davy (CSIRO) Adrian Burton (ARDC) Outline Introduction and trajectory (James Gallapher) OPeNDAP

More information

The Common Framework for Earth Observation Data. US Group on Earth Observations Data Management Working Group

The Common Framework for Earth Observation Data. US Group on Earth Observations Data Management Working Group The Common Framework for Earth Observation Data US Group on Earth Observations Data Management Working Group Agenda USGEO and BEDI background Concise summary of recommended CFEOD standards today Full document

More information

Desarrollo de una herramienta de visualización de datos oceanográficos: Modelos y Observaciones

Desarrollo de una herramienta de visualización de datos oceanográficos: Modelos y Observaciones Desarrollo de una herramienta de visualización de datos oceanográficos: Modelos y Observaciones J. Fernandez, J. Lopez, I. Carlos, F. Jerez, F. Hermosilla, M. Espino SIMO, LIM-UPC, CIIRC FIELD_AC 7 th

More information

New Datasets, Functionality and Future Development. Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama(

New Datasets, Functionality and Future Development. Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama( HYCOM Data Service New Datasets, Functionality and Future Development Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama( Consulting) Roland Schweitzer (Weathertop

More information

Catalog-driven, Reproducible Workflows for Ocean Science

Catalog-driven, Reproducible Workflows for Ocean Science Catalog-driven, Reproducible Workflows for Ocean Science Rich Signell, USGS, Woods Hole, MA, USA Filipe Fernandes, Centro Universidade Monte Serrat, Santos, Brazil. 2015 Boston Light Swim, Aug 15, 7:00am

More information

cdo Data Processing (and Production) Luis Kornblueh, Uwe Schulzweida, Deike Kleberg, Thomas Jahns, Irina Fast

cdo Data Processing (and Production) Luis Kornblueh, Uwe Schulzweida, Deike Kleberg, Thomas Jahns, Irina Fast cdo Data Processing (and Production) Luis Kornblueh, Uwe Schulzweida, Deike Kleberg, Thomas Jahns, Irina Fast Max-Planck-Institut für Meteorologie, DKRZ September 24, 2014 MAX-PLANCK-GESELLSCHAFT Data

More information

Using the Network Common Data Form for storage of atmospheric data

Using the Network Common Data Form for storage of atmospheric data Using the Network Common Data Form for storage of atmospheric data Maarten Plieger Royal Netherlands Meteorological Institute The NetCDF4 data format (Network Common Data Form) is used within the ADAGUC

More information

The NOAA Operational Model Archive and Distribution System (NOMADS)

The NOAA Operational Model Archive and Distribution System (NOMADS) The NOAA Operational Model Archive and Distribution System (NOMADS) Glenn Rutledge Principle Investigator January 2003 Overview Retrospective analysis and model inter-comparison are necessary to verify

More information

The Many Facets of THREDDS Thematic Real-time Environmental Distributed Data Services

The Many Facets of THREDDS Thematic Real-time Environmental Distributed Data Services The Many Facets of THREDDS Thematic Real-time Environmental Distributed Data Services For March 2007 Unidata Policy Committee Meeting Ben Domenico 1 Motivation From the Unidata 2003 proposal: utilizing

More information

NetCDF and HDF5. NASA Earth Science Data Systems Working Group October 20, 2010 New Orleans. Ed Hartnett, Unidata/UCAR, 2010

NetCDF and HDF5. NASA Earth Science Data Systems Working Group October 20, 2010 New Orleans. Ed Hartnett, Unidata/UCAR, 2010 NetCDF and HDF5 NASA Earth Science Data Systems Working Group October 20, 2010 New Orleans Ed Hartnett, Unidata/UCAR, 2010 Unidata Mission: To provide the data services, tools, and cyberinfrastructure

More information

The Logical Data Store

The Logical Data Store Tenth ECMWF Workshop on Meteorological Operational Systems 14-18 November 2005, Reading The Logical Data Store Bruce Wright, John Ward & Malcolm Field Crown copyright 2005 Page 1 Contents The presentation

More information

OPeNDAP: Accessing HYCOM (and other data) remotely

OPeNDAP: Accessing HYCOM (and other data) remotely OPeNDAP: Accessing HYCOM (and other data) remotely Presented at The HYCOM NOPP GODAE Meeting By Peter Cornillon OPeNDAP Inc., Narragansett, RI 02882 7 December 2005 8/25/05 HYCOM NOPP GODAE 1 Acknowledgements

More information

CF-netCDF and CDM. Ethan Davis, John Caron, Ben Domenico, Stefano Nativi* UCAR Unidata Univ of Florence*

CF-netCDF and CDM. Ethan Davis, John Caron, Ben Domenico, Stefano Nativi* UCAR Unidata Univ of Florence* CF-netCDF and CDM Ethan Davis, John Caron, Ben Domenico, Stefano Nativi* UCAR Unidata Univ of Florence* OGC in MetOcean, Toulouse, France, November 2009 CF-netCDF and CDM CF-netCDF CDM/netCDF-java TDS

More information

NetCDF and Scientific Data Durability. Russ Rew, UCAR Unidata ESIP Federation Summer Meeting

NetCDF and Scientific Data Durability. Russ Rew, UCAR Unidata ESIP Federation Summer Meeting NetCDF and Scientific Data Durability Russ Rew, UCAR Unidata ESIP Federation Summer Meeting 2009-07-08 For preserving data, is format obsolescence a non-issue? Why do formats (and their access software)

More information

Implementing a Data Quality Strategy to simplify access to data

Implementing a Data Quality Strategy to simplify access to data IN43D-07 AGU Fall Meeting 2016 Implementing a Quality Strategy to simplify access to data Kelsey Druken, Claire Trenham, Ben Evans, Clare Richards, Jingbo Wang, & Lesley Wyborn National Computational Infrastructure,

More information

Steve Ansari *, Stephen Del Greco, Neal Lott NOAA National Climatic Data Center, Asheville, North Carolina 2. DATA

Steve Ansari *, Stephen Del Greco, Neal Lott NOAA National Climatic Data Center, Asheville, North Carolina 2. DATA 5A.1 THE WEATHER AND CLIMATE TOOLKIT Steve Ansari *, Stephen Del Greco, Neal Lott NOAA National Climatic Data Center, Asheville, North Carolina ABSTRACT The Weather and Climate Toolkit (WCT) is free, platform

More information

The NCAR Community Data Portal

The NCAR Community Data Portal The NCAR Community Data Portal http://cdp.ucar.edu/ QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture. QuickTime and a TIFF (Uncompressed) decompressor are needed to see this

More information

Lynnes, Yang, Hu, Domenico and Enloe Category: Technical Note March Interoperability between OGC CS/W and WCS Protocols

Lynnes, Yang, Hu, Domenico and Enloe Category: Technical Note March Interoperability between OGC CS/W and WCS Protocols Status of this RFC This RFC Technical Note describes a project to provide a catalog search service for the Thematic Realtime Environmental Data Distribution System (THREDDS). Specifically, the project

More information

CESM Projects Using ESMF and NUOPC Conventions

CESM Projects Using ESMF and NUOPC Conventions CESM Projects Using ESMF and NUOPC Conventions Cecelia DeLuca NOAA ESRL/University of Colorado CESM Annual Workshop June 18, 2014 Outline ESMF development update Joint CESM-ESMF projects ESMF applications:

More information

RAMADDA and THREDDS. Projects. Tom Yoksas, John Caron, Ethan Davis 1 Jeff McWhirter 2, Don Murray 3 Matthew Lazzara 4. Unidata Program Center/UCAR 2

RAMADDA and THREDDS. Projects. Tom Yoksas, John Caron, Ethan Davis 1 Jeff McWhirter 2, Don Murray 3 Matthew Lazzara 4. Unidata Program Center/UCAR 2 Use in Support of Two Disparate RAMADDA and THREDDS Projects Tom Yoksas, John Caron, Ethan Davis 1 Jeff McWhirter 2, Don Murray 3 Matthew Lazzara 4 1 Program Center/UCAR 2 UNAVCO 3 NOAA/ESRL/PSD and CIRES

More information

The CEDA Web Processing Service for rapid deployment of earth system data services

The CEDA Web Processing Service for rapid deployment of earth system data services The CEDA Web Processing Service for rapid deployment of earth system data services Stephen Pascoe Ag Stephens Phil Kershaw Centre of Environmental Data Archival 1 1 Overview of CEDA-WPS History first implementation

More information

OGC at KNMI: Current use and plans

OGC at KNMI: Current use and plans OGC at KNMI: Current use and plans 4th Workshop on the use of GIS/OGC standards in meteorology 4 th of March 2013, Reading 1. Ernst de Vreede 2. Maarten Plieger Contents 1. ADAGUC 2. Internal applications

More information

Online intercomparison of models and observations using OGC and community standards

Online intercomparison of models and observations using OGC and community standards Online intercomparison of models and observations using OGC and community standards Alastair Gemmell * Jon Blower Keith Haines Adit Santokhee Reading e-science e Centre, Environmental Systems Science Centre,

More information

THE GEOSS PLATFORM TOWARDS A BIG EO DATA SYSTEM LINKING GLOBAL USERS AND DATA PROVIDERS

THE GEOSS PLATFORM TOWARDS A BIG EO DATA SYSTEM LINKING GLOBAL USERS AND DATA PROVIDERS THE PLATFORM TOWARDS A BIG EO DATA SYSTEM LINKING GLOBAL USERS AND DATA PROVIDERS J. Van Bemmelen (1), P. De Salvo (2), M. Santoro (3), P. Mazzetti (3), G. Colangeli (1), S. Nativi (4) (1) European Space

More information

Data Centre NetCDF Implementation Pilot

Data Centre NetCDF Implementation Pilot Data Centre NetCDF Implementation Pilot Peter Miu EUMETSAT User Conference Oslo, Sep. 2011 Splinter Session, Facilitating Data Access and Utilisation Slide: 1 EUM/OPS/VWG/11/2600 V.1 What is this Pilot

More information

Bruce Wright, John Ward, Malcolm Field, Met Office, United Kingdom

Bruce Wright, John Ward, Malcolm Field, Met Office, United Kingdom The Met Office s Logical Store Bruce Wright, John Ward, Malcolm Field, Met Office, United Kingdom Background are the lifeblood of the Met Office. However, over time, the organic, un-governed growth of

More information

Implementing a Data Quality Strategy to simplify access to data

Implementing a Data Quality Strategy to simplify access to data Implementing a Quality Strategy to simplify access to data Kelsey Druken Implementing a Quality Strategy to simplify access to data Kelsey Druken, Claire Trenham, Lesley Wyborn, Ben Evans National Computational

More information

HYCOM Data Service and Web Outreach

HYCOM Data Service and Web Outreach HYCOM Data Service and Web Outreach A. Srinivasan 1, S. Hankin, P. Cornillon, and E. P. Chassignet 1 Center for Ocean-Atmospheric Prediction Studies, Florida State University phone: (850) 644-7462 fax:

More information

NOAA NextGen IT/Web Services (NGITWS)

NOAA NextGen IT/Web Services (NGITWS) NOAA NextGen IT/Web Services (NGITWS) Robert Bunge (Office of Dissemination) Ryan Solomon (Aviation Weather Center) Steve Olson (Office of Science and Technology) August 24, 2016 ATIEC 2016 Topics Origins

More information

SCSODC: Integrating Ocean Data for Visualization Sharing and Application

SCSODC: Integrating Ocean Data for Visualization Sharing and Application IOP Conference Series: Earth and Environmental Science OPEN ACCESS SCSODC: Integrating Ocean Data for Visualization Sharing and Application To cite this article: C Xu et al 2014 IOP Conf. Ser.: Earth Environ.

More information

Observations and Measurements as a basis for semantic reconciliation between GRIB and netcdf... and some other ideas.

Observations and Measurements as a basis for semantic reconciliation between GRIB and netcdf... and some other ideas. Observations and Measurements as a basis for semantic reconciliation between GRIB and netcdf... and some other ideas. Jeremy Tandy 24 th September 2014 Problem statement: interoperability interoperable

More information

Python: Working with Multidimensional Scientific Data. Nawajish Noman Deng Ding

Python: Working with Multidimensional Scientific Data. Nawajish Noman Deng Ding Python: Working with Multidimensional Scientific Data Nawajish Noman Deng Ding Outline Scientific Multidimensional Data Ingest and Data Management Analysis and Visualization Extending Analytical Capabilities

More information

Unidata and data-proximate analysis and visualization in the cloud

Unidata and data-proximate analysis and visualization in the cloud Unidata and data-proximate analysis and visualization in the cloud Mohan Ramamurthy and Many Unidata Staff 1 June 2017 Modeling in the Cloud Workshop Unidata: A program of the community, by the community,

More information

NetCDF-4: : Software Implementing an Enhanced Data Model for the Geosciences

NetCDF-4: : Software Implementing an Enhanced Data Model for the Geosciences NetCDF-4: : Software Implementing an Enhanced Data Model for the Geosciences Russ Rew, Ed Hartnett, and John Caron UCAR Unidata Program, Boulder 2006-01-31 Acknowledgments This work was supported by the

More information

ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P.

ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. Kushner, D. Waliser, S. Pascoe, A. Stephens, P. Kershaw,

More information

Adapting Software to NetCDF's Enhanced Data Model

Adapting Software to NetCDF's Enhanced Data Model Adapting Software to NetCDF's Enhanced Data Model Russ Rew UCAR Unidata EGU, May 2010 Overview Background What is netcdf? What is the netcdf classic data model? What is the netcdf enhanced data model?

More information

Developing data catalogue extensions for metadata harvesting in GIS

Developing data catalogue extensions for metadata harvesting in GIS University of Bergen Department of Informatics Developing data catalogue extensions for metadata harvesting in GIS Author: André Mossige Long master thesis June 2018 Acknowledgements I would like to thank

More information

Clare Richards, Benjamin Evans, Kate Snow, Chris Allen, Jingbo Wang, Kelsey A Druken, Sean Pringle, Jon Smillie and Matt Nethery. nci.org.

Clare Richards, Benjamin Evans, Kate Snow, Chris Allen, Jingbo Wang, Kelsey A Druken, Sean Pringle, Jon Smillie and Matt Nethery. nci.org. The important role of HPC and data-intensive infrastructure facilities in supporting a diversity of Virtual Research Environments (VREs): working with Climate Clare Richards, Benjamin Evans, Kate Snow,

More information

Meteorology and Python

Meteorology and Python Meteorology and Python desperately trying to forget technical details Claude Gibert, Europython 2011 Background Meteorology - NWP Numerical Weather Prediction ECMWF European Centre for Medium-Range Weather

More information

Ocean Color Data Formats and Conventions:

Ocean Color Data Formats and Conventions: Ocean Color Data Formats and Conventions: NASA's perspective Sean Bailey NASA Goddard Space Flight Center 07 May 2013 International Ocean Color Science Meeting Darmstadt, Germany 1 The Big Picture The

More information

Understanding users workflows

Understanding users workflows Understanding users workflows GloFAS current and future data services Fredrik Wetterhall 1 University of Leeds 6 September 2017 Objective of the talk and exercise 1. Demonstrate how GloFAS data is disseminated

More information

11A.3 INVESTIGATION OF USING HDF5 ARCHIVAL INFORMATION PACKAGES (AIP) TO STORE NASA ECS DATA

11A.3 INVESTIGATION OF USING HDF5 ARCHIVAL INFORMATION PACKAGES (AIP) TO STORE NASA ECS DATA 11A.3 INVESTIGATION OF USING HDF5 ARCHIVAL INFORMATION PACKAGES (AIP) TO STORE NASA ECS DATA MuQun Yang* 1, Ruth Duerr 2, Choonghwan Lee 1 1 The HDF Group, 2 National Snow and Ice Data Center (NSIDC),

More information

NetCDF and Related Interna/onal Standards

NetCDF and Related Interna/onal Standards NetCDF and Related Interna/onal Standards Ben Domenico October 2012 Outline Brief historical context Unidata/partners have established a solid founda/on: Standard data access interfaces enable other Earth

More information

P2.1 Accessing HDF5 data via OPeNDAP. MuQun Yang* 1, Hyo-Kyung Lee 1, James Gallagher 2. I. Introduction

P2.1 Accessing HDF5 data via OPeNDAP. MuQun Yang* 1, Hyo-Kyung Lee 1, James Gallagher 2. I. Introduction P2.1 Accessing HDF5 data via OPeNDAP MuQun Yang* 1, Hyo-Kyung Lee 1, James Gallagher 2 1 The HDF Group, 2 OPeNDAP, Inc. I. Introduction HDF[1] is a set of data formats and software libraries for storing

More information

The Earth System Modeling Framework (and Beyond)

The Earth System Modeling Framework (and Beyond) The Earth System Modeling Framework (and Beyond) Fei Liu NOAA Environmental Software Infrastructure and Interoperability http://www.esrl.noaa.gov/nesii/ March 27, 2013 GEOSS Community ESMF is an established

More information

Toward the Development of a Comprehensive Data & Information Management System for THORPEX

Toward the Development of a Comprehensive Data & Information Management System for THORPEX Toward the Development of a Comprehensive Data & Information Management System for THORPEX Mohan Ramamurthy, Unidata Steve Williams, JOSS Jose Meitin, JOSS Karyn Sawyer, JOSS UCAR Office of Programs Boulder,

More information

ExArch, Edinburgh, March 2014

ExArch, Edinburgh, March 2014 ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. Kushner, D. Waliser, S. Pascoe, A. Stephens, P. Kershaw,

More information

NetCDF- Java Overview. John Caron Oct 29, 2010

NetCDF- Java Overview. John Caron Oct 29, 2010 NetCDF- Java Overview John Caron Oct 29, 2010 Contents Data Models / Shared Dimensions Coordinate Systems Feature Types NetCDF Markup Language (NcML) THREDDS Data Server (TDS) NetCDF- 3 data model MulMdimensional

More information

HDF Product Designer Documentation

HDF Product Designer Documentation HDF Product Designer Documentation Release 1.0.2 The HDF Group May 29, 2015 Contents 1 Introduction 3 2 What s New 5 3 Known Issues 7 4 Getting Started 9 4.1 Download.................................................

More information

Software stack deployment for Earth System Modelling using Spack

Software stack deployment for Earth System Modelling using Spack Software stack deployment for Earth System Modelling using Spack Kim Serradell Maronda (BSC) Sergey Kosukhin (MPI-M) The ESiWACE project has received funding from the European Union s Horizon 2020 research

More information

THREDDS and the THREDDS Data Server (TDS) Don Murray CU/CIRES Slides by Ethan Davis - Unidata

THREDDS and the THREDDS Data Server (TDS) Don Murray CU/CIRES Slides by Ethan Davis - Unidata THREDDS and the THREDDS Data Server (TDS) Don Murray CU/CIRES Slides by Ethan Davis - Unidata THREDDS Data Server (TDS) Web server for scienmfic data (wripen in 100% Java) Can serve any dataset the netcdf-

More information

Leveraging OGC Services in ArcGIS Server. Satish Sankaran, Esri Yingqi Tang, Esri

Leveraging OGC Services in ArcGIS Server. Satish Sankaran, Esri Yingqi Tang, Esri Leveraging OGC Services in ArcGIS Server Satish Sankaran, Esri Yingqi Tang, Esri GIS Creating and Managing Geo Information Products - Proprietary - Open Specifications - Standards Dissemination of Geo

More information

Intro to CMIP, the WHOI CMIP5 community server, and planning for CMIP6

Intro to CMIP, the WHOI CMIP5 community server, and planning for CMIP6 Intro to CMIP, the WHOI CMIP5 community server, and planning for CMIP6 Caroline Ummenhofer, PO Overview - Background on IPCC & CMIP - WHOI CMIP5 server - Available model output - How to access files -

More information

The GEO Discovery and Access Broker

The GEO Discovery and Access Broker WGISS-43 The GEO Discovery and Access Broker S. Nativi (1), M. Santoro (1) (1) Institute of Atmospheric Pollution Research, National Research Council of Italy DOWNSTREAM MIDSTREAM end-users Applications

More information

Part 1: Content model

Part 1: Content model Provläsningsexemplar / Preview TECHNICAL SPECIFICATION ISO/TS 19163-1 First edition 2016-01-15 Geographic information Content components and encoding rules for imagery and gridded data Part 1: Content

More information

Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners

Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners 24th Forum ORAP Cite Scientifique; Lille, France March 26, 2009 Don Middleton National

More information

Oceanography and Ocean Data Assimilation Geospatial Data Grids

Oceanography and Ocean Data Assimilation Geospatial Data Grids Oceanography and Ocean Data Assimilation Geospatial Data Grids Keith Haines Environmental Systems Science Centre (ESSC), Reading University Reading e-science Centre www.resc.reading.ac.uk New technology

More information

Working with Scientific Data in ArcGIS Platform

Working with Scientific Data in ArcGIS Platform Working with Scientific Data in ArcGIS Platform Sudhir Raj Shrestha sshrestha@esri.com Hong Xu hxu@esri.com Esri User Conference, San Diego, CA. July 11, 2017 What we will cover today Scientific Multidimensional

More information

Oceanic Observatory for the Iberian Shelf

Oceanic Observatory for the Iberian Shelf Oceanic Observatory for the Iberian Shelf B.Vila Barcelona, 26th September 2016 Objectives: The Project Improve the oceanic observation at the North Western Iberian coast (meteorological, oceanographical

More information

The EC Presenting a multi-terabyte dataset MWF via ER the web

The EC Presenting a multi-terabyte dataset MWF via ER the web The EC Presenting a multi-terabyte dataset MWF via ER the web Data Management at the BADC Ag Stephens BADC Data Scientist 11 November 2003 Presentation outline An introduction to the BADC. The project

More information

WRF Post-Processing and Visualization

WRF Post-Processing and Visualization NCAR Earth System Laboratory National Center for Atmospheric Research NCAR is Sponsored by NSF and this work is partially supported by the Willis Research Network and the Research Partnership to Secure

More information

Achieving Interoperability using the ArcGIS Platform. Satish Sankaran Roberto Lucchi

Achieving Interoperability using the ArcGIS Platform. Satish Sankaran Roberto Lucchi Achieving Interoperability using the ArcGIS Platform Satish Sankaran Roberto Lucchi Thank you Please fill out the session survey in your mobile app Select [Achieving Interoperability using the ArcGIS Platform]

More information

First experiences of using WC(P)S at ECMWF

First experiences of using WC(P)S at ECMWF Earth Server-2 First experiences of using WC(P)S at ECMWF Julia Wagemann and Stephan Siemen European Centre for Medium-Range Weather Forecasts Workshop on Meteorological Operational Systems #OpenDataWeek

More information

IMOS/AODN ocean portal: tools for data delivery. Roger Proctor, Peter Blain, Sebastien Mancini IMOS

IMOS/AODN ocean portal: tools for data delivery. Roger Proctor, Peter Blain, Sebastien Mancini IMOS IMOS/AODN ocean portal: tools for data delivery Roger Proctor, Peter Blain, Sebastien Mancini IMOS Data from IMOS: The six Nodes Bluewater and Climate Node open ocean focus Five Regional Nodes continental

More information

CERA: Database System and Data Model

CERA: Database System and Data Model CERA: Database System and Data Model Michael Lautenschlager Frank Toussaint World Data Center for Climate (M&D/MPIMET, Hamburg) NINTH WORKSHOP ON METEOROLOGICAL OPERATIONAL SYSTEMS ECMWF, Reading/Berks.,

More information

International Civil Aviation Organization THE SECOND MEETING OF SYSTEM WIDE INFORMATION MANAGEMENT TASK FORCE (SWIM TF/2)

International Civil Aviation Organization THE SECOND MEETING OF SYSTEM WIDE INFORMATION MANAGEMENT TASK FORCE (SWIM TF/2) SWIM TF/2 IP/09 Agenda Item 4 09-12/04/18 International Civil Aviation Organization THE SECOND MEETING OF SYSTEM WIDE INFORMATION MANAGEMENT TASK FORCE (SWIM TF/2) Bangkok, Thailand, 09 12 April 2018 Agenda

More information

GEOSS Common Infrastructure and the Big Data challenges

GEOSS Common Infrastructure and the Big Data challenges 16th Workshop on meteorological operational systems 1-3 March, 2017 GEOSS Common Infrastructure and the Big Data challenges S. Nativi (1), J. Van Bemmelen (2), M. Santoro (1), G. Colangeli (2) O. Ochiai

More information

HPC Performance Advances for Existing US Navy NWP Systems

HPC Performance Advances for Existing US Navy NWP Systems HPC Performance Advances for Existing US Navy NWP Systems Timothy Whitcomb, Kevin Viner Naval Research Laboratory Marine Meteorology Division Monterey, CA Matthew Turner DeVine Consulting, Monterey, CA

More information

The CEDA Archive: Data, Services and Infrastructure

The CEDA Archive: Data, Services and Infrastructure The CEDA Archive: Data, Services and Infrastructure Kevin Marsh Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk with thanks to V. Bennett, P. Kershaw, S. Donegan and the rest of the CEDA Team

More information

CMEMS Service Evolution 21-SE-CALL1. Recommendation and guideline to set up the hardware and software tools to provide HFR data to the HFR Node

CMEMS Service Evolution 21-SE-CALL1. Recommendation and guideline to set up the hardware and software tools to provide HFR data to the HFR Node CMEMS Service Evolution 21-SE-CALL1 Recommendation and guideline to set up the hardware and software tools to provide HFR data to the HFR Node July26th, 2017 1 FOREWORD This document is the first version

More information

The netcdf- 4 data model and format. Russ Rew, UCAR Unidata NetCDF Workshop 25 October 2012

The netcdf- 4 data model and format. Russ Rew, UCAR Unidata NetCDF Workshop 25 October 2012 The netcdf- 4 data model and format Russ Rew, UCAR Unidata NetCDF Workshop 25 October 2012 NetCDF data models, formats, APIs Data models for scienbfic data and metadata - classic: simplest model - - dimensions,

More information

Introduction to HPC Parallel I/O

Introduction to HPC Parallel I/O Introduction to HPC Parallel I/O Feiyi Wang (Ph.D.) and Sarp Oral (Ph.D.) Technology Integration Group Oak Ridge Leadership Computing ORNL is managed by UT-Battelle for the US Department of Energy Outline

More information

The NCI High Performance Computing (HPC) and High Performance Data (HPD) Platform to Support the Analysis of Petascale Environmental Data Collections

The NCI High Performance Computing (HPC) and High Performance Data (HPD) Platform to Support the Analysis of Petascale Environmental Data Collections ESSI 2015-8273 The NCI High Performance Computing (HPC) and High Performance Data (HPD) Platform to Support the Analysis of Petascale Environmental Data Collections Ben Evans 1, Lesley Wyborn 1, Tim Pugh

More information

Writing NetCDF Files: Formats, Models, Conventions, and Best Practices. Overview

Writing NetCDF Files: Formats, Models, Conventions, and Best Practices. Overview Writing NetCDF Files: Formats, Models, Conventions, and Best Practices Russ Rew, UCAR Unidata June 28, 2007 1 Overview Formats, conventions, and models NetCDF-3 limitations NetCDF-4 features: examples

More information

Instituting an observation database

Instituting an observation database Instituting an observation database capability in the NCEP GSI Tom Hamill, Jeff Whitaker, Scott Gregory NOAA / ESRL Physical Sciences Division Presentation to DAOS, Exeter England, April 2016 Our intent:

More information

Instituting an observation database (ODB) capability in the GSI

Instituting an observation database (ODB) capability in the GSI Instituting an observation database (ODB) capability in the GSI Jeff Whitaker, Scott Gregory, and Tom Hamill NOAA / ESRL Physical Sciences Division Presentation to Blueprints for Next-Generation Data Assimilation

More information

Data Serving Climate Simulation Science at the NASA Center for Climate Simulation (NCCS)

Data Serving Climate Simulation Science at the NASA Center for Climate Simulation (NCCS) Data Serving Climate Simulation Science at the NASA Center for Climate Simulation (NCCS) MSST2011, May 24-25, 2011 Ellen Salmon ( Ellen.Salmon@nasa.gov ) High Performance Computing, Code 606.2 Computational

More information

HDF Update. Elena Pourmal The HDF Group. October 16, 2008 IDL User Group Meeting 1

HDF Update. Elena Pourmal The HDF Group. October 16, 2008 IDL User Group Meeting 1 HDF Update Elena Pourmal The HDF Group October 16, 2008 IDL User Group Meeting 1 The HDF Group The HDF Group is a not-for-profit company with its mission focused on the support and growth of the HDF technologies

More information

Interactive Web Mapping: Overview

Interactive Web Mapping: Overview Interactive Web Mapping: Overview Overview of how geospatial data is formatted requested supplied consumed by/for web technologies 2 Definitions Analysis exploring and modeling geospatial phenomena Mapping

More information

Modeling groups and Data Center Requirements. Session s Keynote. Sébastien Denvil, CNRS, Institut Pierre Simon Laplace (IPSL)

Modeling groups and Data Center Requirements. Session s Keynote. Sébastien Denvil, CNRS, Institut Pierre Simon Laplace (IPSL) Modeling groups and Data Center Requirements. Session s Keynote. Sébastien Denvil, CNRS, Institut Pierre Simon Laplace (IPSL) Outline Major constraints (requirements' DNA) Modeling center requirements/constraints

More information

Database Web Portal Version 1

Database Web Portal Version 1 Database Web Portal Version 1 Deliverable D6.5 Issue 0.2 Due date of deliverable: 01 Dec 2014 Actual submission date: 05 Dec 2014 SEN3APP Processing Lines And Operational Services Combining Sentinel And

More information

J1.6 MONITORING AND ANALYZING THE GLOBAL OCEAN OBSERVING SYSTEM WITH THE OBSERVING SYSTEM MONITORING CENTER

J1.6 MONITORING AND ANALYZING THE GLOBAL OCEAN OBSERVING SYSTEM WITH THE OBSERVING SYSTEM MONITORING CENTER J1.6 MONITORING AND ANALYZING THE GLOBAL OCEAN OBSERVING SYSTEM WITH THE OBSERVING SYSTEM MONITORING CENTER Kevin M. O'Brien 1*,S. Hankin 2, R. Schweitzer 3, K. Kern 4, M. Little 4,T. Habermann 5, N. Auerbach

More information

The Integration of Grid Technology with OGC Web Services (OWS) in NWGISS for NASA EOS Data

The Integration of Grid Technology with OGC Web Services (OWS) in NWGISS for NASA EOS Data The Integration of Grid Technology with OGC Web Services (OWS) in NWGISS for NASA EOS Data Liping Di, Aijun Chen, Wenli Yang and Peisheng Zhao achen6@gmu.edu; achen@laits.gmu.edu Lab for Advanced Information

More information

Data Management Components for a Research Data Archive

Data Management Components for a Research Data Archive Data Management Components for a Research Data Archive Steven Worley and Bob Dattore Scientific Computing Division Computational and Information Systems Laboratory National Center for Atmospheric Research

More information

CEOS Water Portal Project Implementation Plan

CEOS Water Portal Project Implementation Plan CEOS Water Portal Project Implementation Plan March 10th, 2015 Version 2.0 Revision History Version Date Page Reason Comment 1.0 Mar 2, 2012 - (v1.0 released) 1.1 Feb 2,2013 6 Update to incorporate new

More information

NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION S SCIENTIFIC DATA STEWARDSHIP PROGRAM

NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION S SCIENTIFIC DATA STEWARDSHIP PROGRAM J2.3A NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION S SCIENTIFIC DATA STEWARDSHIP PROGRAM John J. Bates * NOAA National Climatic Center, Asheville, North Carolina Richard G. Reynolds Office of Systems

More information

NetCDF-4: A New Data Model, Programming Interface, and Format Using HDF5

NetCDF-4: A New Data Model, Programming Interface, and Format Using HDF5 NetCDF-4: A New Data Model, Programming Interface, and Format Using HDF5 Russ Rew, Ed Hartnett, John Caron UCAR Unidata Program Center Mike Folk, Robert McGrath, Quincey Kozial NCSA and The HDF Group,

More information

Meta4+CMIP5. Bryan Lawrence, Gerry Devine and many many others

Meta4+CMIP5. Bryan Lawrence, Gerry Devine and many many others Meta4+CMIP5 Bryan Lawrence, Gerry Devine and many many others, Outline A metadata taxonomy: a place for everyone (and everyone in their place :-) Introduction to Key Metafor Concepts CIM Metafor and IS-ENES

More information

WCS2.1 Met-Ocean. 16 th Workshop on Operational Systems. Crown copyright Met Office

WCS2.1 Met-Ocean. 16 th Workshop on Operational Systems. Crown copyright Met Office WCS2.1 Met-Ocean 16 th Workshop on Operational Systems Crown copyright Met Office A Revolutionary Approach to Multi- Dimensional Data Access of Gridded Datasets Using Current and Proposed OGC Web Coverage

More information

European and international background

European and international background IS-ENES2 General Assembly 11-13th June 2014 Barcelona European and international background Metadata and ES-DOC (including statistical downscaling) Sébastien Denvil, Mark Greenslade, Allyn Treshansky,

More information

CMIP5 Datenmanagement erste Erfahrungen

CMIP5 Datenmanagement erste Erfahrungen CMIP5 Datenmanagement erste Erfahrungen Dr. Michael Lautenschlager Deutsches Klimarechenzentrum Helmholtz Open Access Webinare zu Forschungsdaten Webinar 18-17.01./28.01.14 CMIP5 Protocol +Timeline Taylor

More information

Putting Your Data on the Map

Putting Your Data on the Map Putting Your Data on the Map Cyndy Chandler WHOI Marine Chemistry Department Woods Hole Data Mongers / Ocean Informatics Meeting 4 April 2007 - WHOI Finding the OI in WHOI What is Ocean Informatics? The

More information

GEMS: Global Earth-system Monitoring using Satellite & in-situ Data

GEMS: Global Earth-system Monitoring using Satellite & in-situ Data GEMS: Global Earth-system Monitoring using Space and in-situ data Introduction to GEMS 2006 Assembly GMES Integrated Project, 12.5MEuro, 30 Institutes, 14 Countries www.ecmwf.int/research/eu_projects/gems

More information

On the Processing of Extreme Scale Datasets in the Geosciences

On the Processing of Extreme Scale Datasets in the Geosciences Chapter 1 On the Processing of Extreme Scale Datasets in the Geosciences Sangmi Lee Pallickara, Matthew Malensek, and Shrideep Pallickara Department of Computer Science Colorado State University, Fort

More information

PRISM Project for Integrated Earth System Modelling An Infrastructure Project for Climate Research in Europe funded by the European Commission

PRISM Project for Integrated Earth System Modelling An Infrastructure Project for Climate Research in Europe funded by the European Commission PRISM Project for Integrated Earth System Modelling An Infrastructure Project for Climate Research in Europe funded by the European Commission under Contract EVR1-CT2001-40012 The VTK_Mapper Application

More information

Future Plans for Pride Observing System Web Services

Future Plans for Pride Observing System Web Services Future Plans for Pride Observing System Web Services Joe Sirott Nancy Soreide,, Donald Denbo PMEL/NOAA 1 Outline Stuff that we ve done Stuff that we d like to do 2 Dapper Web server that provides distributed

More information

HDF Product Designer Documentation

HDF Product Designer Documentation HDF Product Designer Documentation Release 1.5.0 The HDF Group May 31, 2017 Contents 1 Contents: 3 1.1 Getting Started.............................................. 3 1.2 Usage...................................................

More information