Instituting an observation database

Size: px
Start display at page:

Download "Instituting an observation database"

Transcription

1 Instituting an observation database capability in the NCEP GSI Tom Hamill, Jeff Whitaker, Scott Gregory NOAA / ESRL Physical Sciences Division Presentation to DAOS, Exeter England, April 2016

2 Our intent: an observation database (ODB) capability for the GSI. In these file(s) would be: Observations. Metadata (time stamp, QC flags, lat, lon, observation error assigned). O minus F (including spread if ensemble DA used). O minus A. Bias correction information. Usage flags. Such files would be easily share able and in user friendly formats for external use.

3 Current NCEP observations data flow (Dennis Keyser, from 2013 DTC GSI tutorial) Bottom line: it s complicated.

4 Vision for how this might work in JEDAI (GSI code refactor) GTS (and other sources) Observation Processing read obs (bufr, HDF, netcdf, ascii) set R basic QC Observation Database Manager Relational Database (like ODB), and/or hierarchical dataset (HDF5, netcdf) (replaces BUFR tanks and dump files) O, R, metadata O minus F, O minus A, QC, Observer read background forecast(s) data thinning/channel selection interpolation forward operator background check QC Compute O F, O A Solver (hybrid EnVar, EnKF)

5 Our project: global reanalysis, ~2000 current. How to add ODB like capability for this? Several options: Adapt branch of code for NCEP CFS reanalysis (> 5 years old) that provides this to the current GSI. Software old, display infrastructure (GRaDS based) old. Totally refactor the GSI and include ODB (see upcoming slide). Patch existing GSI system code to produce the observation database capability we need for this project (see upcoming slide). being mindful of longer term refactor and making code useful for this.

6 Patching the GSI to provide ODB capability while we await the GSI refactor. Can be done in the short term for next GEFS reanalysis/reforecast without changing NCEP operational workflow. Enables easier access to observations and assimilation feedback information for monitoring, diagnosis, research. Amounts to pre/post processing input BUFR, output GSI diagnostic files.

7 GSI data assimilation system BUFR (prepbufr, radiance bufr, satwnd bufr, gpsro bufr ) analyses GSI diagnostic files (containing obs ingested by GSI, departures, QC, bias correction info) (the current system)

8 GSI data assimilation system python utilities: (1) bufr2nc (outputs BUFR as netcdf file) (2) merge_gsidiag_bufr2nc (matches records in GSI diagnostic file with original BUFR, adds depatures, ens. spread in ob space, QC, other DA info) (3) new_obs_2nc (various codes tailored to particular research data sets) nc2prepbufr (convert back to GSI compatible BUFR) with proposed modifications BUFR (prepbufr, radiance bufr, satwnd bufr, gpsro bufr ) analyses GSI diagnostic files (containing obs ingested by GSI, departures, QC, bias correction info) Observation database Other observation sources (field experiments, research instruments, etc.) OpenDAP data server, access to research community

9 Conclusions We re putting together an interim ODB capability into the GSI while we await a more complete JEDAI code refactor. We re at the early stages and are happy to learn from the experiences of others. The current design is necessitated by a reanalysis project where we must deliver quickly, but we d still like to make as much of our software re usable as possible.

10 Fit of short term forecasts to observations can be revealing of DA system characteristics. from Dee et al. 2011,

11 Fit of other quantities (here bias corrections) can be revealing also. from Dee et al. 2011,

12 Fit of observations to longer lead forecasts can also be revealing. With stochastic physics and verification against analysis, ECMWF ensemble appears to be underspread (left most column). Verifying against raobs or AMSU A channel 5, one has the impression that the ensemble is still under spread. Yamaguchi et al. 2016,

13 ncdump h of a sample netcdf file with bufr data

14 Sample python code for reading in netcdf ODB type file and generating basic statistics.

15 Demonstration plot

Instituting an observation database (ODB) capability in the GSI

Instituting an observation database (ODB) capability in the GSI Instituting an observation database (ODB) capability in the GSI Jeff Whitaker, Scott Gregory, and Tom Hamill NOAA / ESRL Physical Sciences Division Presentation to Blueprints for Next-Generation Data Assimilation

More information

EnKF Fundamentals (2b): Applications

EnKF Fundamentals (2b): Applications 2015 EnKF Community Tutorial August 13-14, 2015. Boulder, CO EnKF Fundamentals (2b): Applications Kathryn Newman Ming Hu, and Chunhua Zhou Developmental Testbed Center (DTC) Outline EnKF fundamentals (1):

More information

Observation feedback archiving in MARS. Acknowledgement:

Observation feedback archiving in MARS. Acknowledgement: Observation feedback archiving in MARS Piotr Kuchta Data and Services Section, ECMWF P.Kuchta@ecmwf.int Acknowledgement: Slide 1 Manuel Fuentes, Baudouin Rault, Erik Andersson, Anne Fouilloux, Lars Isaksen,

More information

GSI Fundamentals (5): Review and Applications

GSI Fundamentals (5): Review and Applications 27 GSI/EnKF Community Tutorial July -4, 27, College Park, MD GSI Fundamentals (5): Review and Applications Jeff Beck, Ming Hu, Hui Shao2, Chunhua Zhou2, and Kathryn Newman2 Developmental Testbed Center

More information

Uniform Resource Locator Wide Area Network World Climate Research Programme Coupled Model Intercomparison

Uniform Resource Locator Wide Area Network World Climate Research Programme Coupled Model Intercomparison Glossary API Application Programming Interface AR5 IPCC Assessment Report 4 ASCII American Standard Code for Information Interchange BUFR Binary Universal Form for the Representation of meteorological

More information

Metview s new Python interface

Metview s new Python interface Metview s new Python interface Workshop on developing Python frameworks for earth system sciences. ECMWF, 2018 Iain Russell Development Section, ECMWF Thanks to Sándor Kertész Fernando Ii Stephan Siemen

More information

GSI fundamentals (4): Background Error Covariance and Observation Error

GSI fundamentals (4): Background Error Covariance and Observation Error 27 GSI Community Tutorial July 2, 27, College Park, MD GSI fundamentals (4): Background Error Covariance and Observation Error Ming Hu Developmental Testbed Center Outline GSI fundamentals (): Setup and

More information

Introduction to NetCDF

Introduction to NetCDF Introduction to NetCDF NetCDF is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. First released in 1989.

More information

Collabora've Development

Collabora've Development Collabora've Development Ricardo Todling NASA GMAO 2013 Joint DTC- EMC- JCSDA GSI Workshop 1 In the Beginning NCEP: SSI (late 1980 s) Spectral formula'on of background error cov Direct assimila'on of radiances

More information

The NOAA Operational Model Archive and Distribution System (NOMADS)

The NOAA Operational Model Archive and Distribution System (NOMADS) The NOAA Operational Model Archive and Distribution System (NOMADS) Glenn Rutledge Principle Investigator January 2003 Overview Retrospective analysis and model inter-comparison are necessary to verify

More information

cdo Data Processing (and Production) Luis Kornblueh, Uwe Schulzweida, Deike Kleberg, Thomas Jahns, Irina Fast

cdo Data Processing (and Production) Luis Kornblueh, Uwe Schulzweida, Deike Kleberg, Thomas Jahns, Irina Fast cdo Data Processing (and Production) Luis Kornblueh, Uwe Schulzweida, Deike Kleberg, Thomas Jahns, Irina Fast Max-Planck-Institut für Meteorologie, DKRZ September 24, 2014 MAX-PLANCK-GESELLSCHAFT Data

More information

eccharts and Metview 4 2 new visualisation systems at ECMWF

eccharts and Metview 4 2 new visualisation systems at ECMWF eccharts and Metview 4 2 new visualisation systems at ECMWF Sylvie Lamy-Thépaut Meteorological Visualisation Section ECMWF Slide 1 eccharts and Metview 4 eccharts Web application - Response Time, Monitoring

More information

Observa(on Processing. Nancy Collins or

Observa(on Processing. Nancy Collins or Observa(on Processing Nancy Collins nancy@ucar.edu or dart@ucar.edu Roadmap What s in an Observa(on Provided tools and capabili(es Observa(on sources Observa(on and representa(veness error Types, Kinds,

More information

WRF-NMM Standard Initialization (SI) Matthew Pyle 8 August 2006

WRF-NMM Standard Initialization (SI) Matthew Pyle 8 August 2006 WRF-NMM Standard Initialization (SI) Matthew Pyle 8 August 2006 1 Outline Overview of the WRF-NMM Standard Initialization (SI) package. More detailed look at individual SI program components. SI software

More information

Progress in the assimilation of GPS-RO at ECMWF

Progress in the assimilation of GPS-RO at ECMWF Progress in the assimilation of GPS-RO at ECMWF Sean Healy Mike Rennie, Milan Dragosavac, Paul Poli, Carla Cardinali, Peter Bauer. Impact of GPS-RO The impact of GPS-RO NWP and climate reanalysis applications

More information

Metview 5.0 and Beyond, to its Pythonic Future

Metview 5.0 and Beyond, to its Pythonic Future Metview 5.0 and Beyond, to its Pythonic Future MOS 2017, ECMWF Iain Russell, Fernando Ii, Sándor Kertész, Stephan Siemen Development Section, ECMWF ECMWF March 02, 2017 What is Metview? Workstation software

More information

Community Tools (1) - PrepBUFR/BUFR: Basic tools, NCEP data, and BUFR table

Community Tools (1) - PrepBUFR/BUFR: Basic tools, NCEP data, and BUFR table 2012 GSI Summer Tutorial, 22 Aug. 2012, Boulder, CO Community Tools (1) - PrepBUFR/BUFR: Basic tools, NCEP data, and BUFR table Ruifang Li NCAR/MMM Topics covered NCEP observation data Observation data

More information

Distributed Online Data Access and Analysis

Distributed Online Data Access and Analysis Distributed Online Data Access and Analysis Ruixin Yang George Mason University Slides from SIESIP Partners and from NOMADS PI, Glenn K. Rutledge of US NCDC on NOMADS SIESIP: Seasonal-to-Interannual Earth

More information

SAPP: a new scalable acquisition and pre-processing system at ECMWF

SAPP: a new scalable acquisition and pre-processing system at ECMWF from Newsletter Number 140 Summer 2014 COMPUTING SAPP: a new scalable acquisition and pre-processing system at ECMWF Salem Alkait/iStock/Thinkstock doi:10.21957/3ylomoa6 This article appeared in the Computing

More information

Introduction to Metview

Introduction to Metview Introduction to Metview Fernando Ii, Iain Russell, Sándor Slide Kertész 1 Development Section - ECMWF 1 What is Metview? Retrieve/manipulate/visualise meteorological data Working environment for operational

More information

HARMONIE ATOVS data assimilation and coordinated impact study

HARMONIE ATOVS data assimilation and coordinated impact study HARMONIE ATOVS data assimilation and coordinated impact study ALADIN / HIRLAM 23rd Workshop / All-Staff Meeting Reykjavik, Iceland, 15-19 April, 2013 Magnus Lindskog, Mats Dahlbom, Sigurdur Thorsteinsson,

More information

The inclusion of cloudy radiances in the NCEP GSI analysis system

The inclusion of cloudy radiances in the NCEP GSI analysis system he inclusion of cloudy radiances in the NCEP GSI analysis system 1,2,3 Min-Jeong Kim 1 Fuzhong Weng and 2 John Derber 1.NOAA/NESDIS/SAR 2. Joint Center for Satellite Data Assimilation (JCSDA) 3. CIRA,

More information

EnKF Fundamentals (1): Configuration and Run

EnKF Fundamentals (1): Configuration and Run 2017 Joint DTC-EMC-JCSDA GSI-EnKF Tutorial July 11-14, 2017, NCWCP, College Park, MD EnKF Fundamentals (1): Configuration and Run Chunhua Zhou * Kathryn Newman * and Ming Hu ** Developmental Testbed Center

More information

Bias correction of satellite data at ECMWF

Bias correction of satellite data at ECMWF Bias correction of satellite data at ECMWF Thomas Auligne Dick Dee, Graeme Kelly, Tony McNally ITSC XV - October 2006 Slide 1 Motivation for an adaptive system Simplify the bias correction process of manual

More information

And now for something completely different

And now for something completely different And now for something completely different (data management?) HYCOM Data Management & Services Ashwanth Srinivasan (RSMAS) Steve Hankin (PMEL) A community of of contributors, including Peter Peter Cornillon,

More information

Metview and Python - what they can do for each other

Metview and Python - what they can do for each other Metview and Python - what they can do for each other Workshop on Python for Earth System Sciences, ECMWF Iain Russell, Fernando Ii, Sándor Kertész, Stephan Siemen Development Section, ECMWF ECMWF November

More information

File Formats and Pre-Processing

File Formats and Pre-Processing File Formats and Pre-Processing File Formats Pre-processing Tools Useful Links Presenter: Tara Jensen Supported File Formats Forecasts GRIB GRIdded Binary file format (*version 1) NetCDF MET specific network

More information

Analysis Methods in Atmospheric and Oceanic Science

Analysis Methods in Atmospheric and Oceanic Science Analysis Methods in Atmospheric and Oceanic Science AOSC 652 HDF & NetCDF files; Regression; File Compression & Data Access Week 11, Day 1 Today: Data Access for Projects; HDF & NetCDF Wed: Multiple Linear

More information

Observational DataBase (ODB*) and its usage at ECMWF

Observational DataBase (ODB*) and its usage at ECMWF Observational DataBase (ODB*) and its usage at ECMWF anne.fouilloux@ecmwf.int Slide 1 *ODB has been developed and maintained by Sami Saarinen Slide 1 Outline Observational usage over the past decades at

More information

NetCDF and HDF5. NASA Earth Science Data Systems Working Group October 20, 2010 New Orleans. Ed Hartnett, Unidata/UCAR, 2010

NetCDF and HDF5. NASA Earth Science Data Systems Working Group October 20, 2010 New Orleans. Ed Hartnett, Unidata/UCAR, 2010 NetCDF and HDF5 NASA Earth Science Data Systems Working Group October 20, 2010 New Orleans Ed Hartnett, Unidata/UCAR, 2010 Unidata Mission: To provide the data services, tools, and cyberinfrastructure

More information

Metview BUFR Tutorial. Meteorological Visualisation Section Operations Department ECMWF

Metview BUFR Tutorial. Meteorological Visualisation Section Operations Department ECMWF Meteorological Visualisation Section Operations Department ECMWF 05/03/2015 This tutorial was tested with Metview version 4.3.0 and some features might not work for previous versions. Copyright 2015 European

More information

A brief introduction 1 to retrieving ERA Interim via the web and webapi

A brief introduction 1 to retrieving ERA Interim via the web and webapi A brief introduction 1 to retrieving ERA Interim via the web and webapi Adrian Tompkins (ICTP), Email: tompkins@ictp.it Last updated May 9, 2017 1 Analysis and Reanalysis Overview For a full description

More information

The C3S Climate Data Store and its upcoming use by CAMS

The C3S Climate Data Store and its upcoming use by CAMS Atmosphere The C3S Climate Data Store and its upcoming use by CAMS Miha Razinger, ECMWF thanks to Angel Alos, Baudouin Raoult, Cedric Bergeron and the CDS contractors Atmosphere What are C3S and CDS? The

More information

Unifying Verification through a Python-wrapped Suite of Tools

Unifying Verification through a Python-wrapped Suite of Tools Support for MET is provided by NOAA, US Air Force, NSF and NCAR through the Developmental Testbed Center (DTC) Unifying Verification through a Python-wrapped Suite of Tools Tara Jensen, John Halley Gotway,

More information

Meteorology and Python

Meteorology and Python Meteorology and Python desperately trying to forget technical details Claude Gibert, Europython 2011 Background Meteorology - NWP Numerical Weather Prediction ECMWF European Centre for Medium-Range Weather

More information

HDF Product Designer: A tool for building HDF5 containers with granule metadata

HDF Product Designer: A tool for building HDF5 containers with granule metadata The HDF Group HDF Product Designer: A tool for building HDF5 containers with granule metadata Lindsay Powers Aleksandar Jelenak, Joe Lee, Ted Habermann The HDF Group Data Producer s Conundrum 2 HDF Features

More information

AAPP status report and preparations for processing METOP data

AAPP status report and preparations for processing METOP data AAPP status report and preparations for processing METOP data Nigel C Atkinson *, Pascal Brunel, Philippe Marguinaud and Tiphaine Labrot * Met Office, Exeter, UK Météo-France, Centre de Météorologie Spatiale,

More information

11A.3 INVESTIGATION OF USING HDF5 ARCHIVAL INFORMATION PACKAGES (AIP) TO STORE NASA ECS DATA

11A.3 INVESTIGATION OF USING HDF5 ARCHIVAL INFORMATION PACKAGES (AIP) TO STORE NASA ECS DATA 11A.3 INVESTIGATION OF USING HDF5 ARCHIVAL INFORMATION PACKAGES (AIP) TO STORE NASA ECS DATA MuQun Yang* 1, Ruth Duerr 2, Choonghwan Lee 1 1 The HDF Group, 2 National Snow and Ice Data Center (NSIDC),

More information

Writing NetCDF Files: Formats, Models, Conventions, and Best Practices. Overview

Writing NetCDF Files: Formats, Models, Conventions, and Best Practices. Overview Writing NetCDF Files: Formats, Models, Conventions, and Best Practices Russ Rew, UCAR Unidata June 28, 2007 1 Overview Formats, conventions, and models NetCDF-3 limitations NetCDF-4 features: examples

More information

Remote Data Access with OPeNDAP. Dr. Dennis Heimbigner Unidata netcdf Workshop October 25, 2012

Remote Data Access with OPeNDAP. Dr. Dennis Heimbigner Unidata netcdf Workshop October 25, 2012 Remote Data Access with OPeNDAP Dr. Dennis Heimbigner Unidata netcdf Workshop October 25, 2012 Overview What are OPeNDAP and DAP? What is a Client-Server Architecture Why is Remote Data Access Useful?

More information

Metview s new Python interface first results and roadmap for further developments

Metview s new Python interface first results and roadmap for further developments Metview s new Python interface first results and roadmap for further developments EGOWS 2018, ECMWF Iain Russell Development Section, ECMWF Thanks to Sándor Kertész Fernando Ii Stephan Siemen ECMWF October

More information

ECMWF New Users Metview Tutorial

ECMWF New Users Metview Tutorial ECMWF New Users Metview Tutorial Author: Date: URL: Iain Russell 06-Mar-2014 08:43 https://software.ecmwf.int/wiki/display/metv/ecmwf+new+users+metview+tutorial 1 of 12 Table of Contents 1 Preparation

More information

Python: Working with Multidimensional Scientific Data. Nawajish Noman Deng Ding

Python: Working with Multidimensional Scientific Data. Nawajish Noman Deng Ding Python: Working with Multidimensional Scientific Data Nawajish Noman Deng Ding Outline Scientific Multidimensional Data Ingest and Data Management Analysis and Visualization Extending Analytical Capabilities

More information

Metview 4 ECMWF s latest generation meteorological workstation

Metview 4 ECMWF s latest generation meteorological workstation Metview 4 ECMWF s latest generation meteorological workstation Iain Russell, Stephan Siemen, Fernando Ii, Sándor Kertész, Sylvie Lamy-Thépaut, Vesa Karhila Version 4 builds on the flexible and proven modular

More information

R. James Purser and Xiujuan Su. IMSG at NOAA/NCEP/EMC, College Park, MD, USA.

R. James Purser and Xiujuan Su. IMSG at NOAA/NCEP/EMC, College Park, MD, USA. A new observation error probability model for Nonlinear Variational Quality Control and applications within the NCEP Gridpoint Statistical Interpolation. R. James Purser and Xiujuan Su IMSG at NOAA/NCEP/EMC,

More information

Amara's law : Overestimating the effects of a technology in the short run and underestimating the effects in the long run

Amara's law : Overestimating the effects of a technology in the short run and underestimating the effects in the long run Cloud Archiving and Data Mining: Operational and Research Examples John Horel*, Brian Blaylock, Chris Galli* Department of Atmospheric Sciences University of Utah john.horel@utah.edu *Synoptic Data Corporation

More information

Ensemble Kalman Filter

Ensemble Kalman Filter NOAA Ensemble Kalman Filter Beta Release v1.0 compatible with GSI community release v3.3 User s Guide January 2015 Developmental Testbed Center National Center for Atmospheric Research National Centers

More information

GSI Setup, Run and Namelist

GSI Setup, Run and Namelist GSI Setup, Run and Namelist Hui Shao GSI Community Tutorial, June 28-30, 2010, Boulder, CO Outline Installing GSI Running GSI Basic runtime options This talk is tailored based on GSI Community Release

More information

New Datasets, Functionality and Future Development. Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama(

New Datasets, Functionality and Future Development. Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama( HYCOM Data Service New Datasets, Functionality and Future Development Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama( Consulting) Roland Schweitzer (Weathertop

More information

Software Infrastructure for Data Assimilation: Object Oriented Prediction System

Software Infrastructure for Data Assimilation: Object Oriented Prediction System Software Infrastructure for Data Assimilation: Object Oriented Prediction System Yannick Trémolet ECMWF Blueprints for Next-Generation Data Assimilation Systems, Boulder, March 2016 Why OOPS? Y. Trémolet

More information

Anne Fouilloux. Fig. 1 Use of observational data at ECMWF since CMA file structure.

Anne Fouilloux. Fig. 1 Use of observational data at ECMWF since CMA file structure. ODB (Observational Database) and its usage at ECMWF Anne Fouilloux Abstract ODB stands for Observational DataBase and has been developed at ECMWF since mid-1998 by Sami Saarinen. The main goal of ODB is

More information

Downscaling tools for adapting climate predictions to the user's needs: Challenges for their integration into climate services

Downscaling tools for adapting climate predictions to the user's needs: Challenges for their integration into climate services Downscaling tools for adapting climate predictions to the user's needs: Challenges for their integration into climate services A.S. Cofiño, J.M. Gutiérrez, J. Fernández, J. Bedia, M. Vega, S. Herrera,

More information

OGC at KNMI: Current use and plans

OGC at KNMI: Current use and plans OGC at KNMI: Current use and plans 4th Workshop on the use of GIS/OGC standards in meteorology 4 th of March 2013, Reading 1. Ernst de Vreede 2. Maarten Plieger Contents 1. ADAGUC 2. Internal applications

More information

Python in the Copernicus Climate Change Service

Python in the Copernicus Climate Change Service Python in the Copernicus Service Gionata Biavati C D S : A n e w w a y o f s e r v i n g d a t a Copernicus Services (C3S) (https://climate.copernicus.eu) is providing the Data Store (CDS) (https://cds.climate.copernicus.eu)

More information

SURFEX LDAS March 2012

SURFEX LDAS March 2012 SURFEX LDAS March 2012 Alina Barbu Combined assimilation of satellite-derived soil moisture and LAI 2 Motivation of our work GEOLAND 2 project Land Carbon Information Service (LCIS) on vegetation/land

More information

First experiences of using WC(P)S at ECMWF

First experiences of using WC(P)S at ECMWF Earth Server-2 First experiences of using WC(P)S at ECMWF Julia Wagemann and Stephan Siemen European Centre for Medium-Range Weather Forecasts Workshop on Meteorological Operational Systems #OpenDataWeek

More information

OPeNDAP: Accessing HYCOM (and other data) remotely

OPeNDAP: Accessing HYCOM (and other data) remotely OPeNDAP: Accessing HYCOM (and other data) remotely Presented at The HYCOM NOPP GODAE Meeting By Peter Cornillon OPeNDAP Inc., Narragansett, RI 02882 7 December 2005 8/25/05 HYCOM NOPP GODAE 1 Acknowledgements

More information

GSI Fundamentals (2) Run and Namelist

GSI Fundamentals (2) Run and Namelist 2 GSI Community Tutorial June 29-July, 2, Boulder, CO GSI Fundamentals (2) Run and Namelist Ming Hu and Hui Shao Developmental Testbed Center Outlines GSI fundamentals (): Setup and Compilation GSI fundamentals

More information

Data Centre NetCDF Implementation Pilot

Data Centre NetCDF Implementation Pilot Data Centre NetCDF Implementation Pilot Peter Miu EUMETSAT User Conference Oslo, Sep. 2011 Splinter Session, Facilitating Data Access and Utilisation Slide: 1 EUM/OPS/VWG/11/2600 V.1 What is this Pilot

More information

ECMWF contribution to the SMOS mission

ECMWF contribution to the SMOS mission contribution to the SMOS mission J. Muñoz Sabater, P. de Rosnay, M. Drusch & G. Balsamo Monitoring Assimilation Remote Sensing and Modeling of Surface Properties 09-11 June 2009 slide 1 Outline Global

More information

The Lidar-Radar Open Software Environment (LROSE) : Progress and Plans

The Lidar-Radar Open Software Environment (LROSE) : Progress and Plans The Lidar-Radar Open Software Environment (LROSE) : Progress and Plans Michael M. Bell Colorado State University Wen-Chau Lee and Mike Dixon NCAR Research supported by NSF SI2-SSI Award ACI-1661663 LROSE:

More information

Advances in Time-Parallel Four Dimensional Data Assimilation in a Modular Software Framework

Advances in Time-Parallel Four Dimensional Data Assimilation in a Modular Software Framework Advances in Time-Parallel Four Dimensional Data Assimilation in a Modular Software Framework Brian Etherton, with Christopher W. Harrop, Lidia Trailovic, and Mark W. Govett NOAA/ESRL/GSD 28 October 2016

More information

Report on the COPE technical meeting held at ECMWF, Reading 9-12, June 2014

Report on the COPE technical meeting held at ECMWF, Reading 9-12, June 2014 1 Background Report on the COPE technical meeting held at ECMWF, Reading 9-12, June 2014 Alena Trojáková The ECMWF initiated the Continuous Observation Processing Environment (COPE) project to support

More information

BUFR/PrepBUFR File Processing

BUFR/PrepBUFR File Processing 2011 BUFR Tutorial, 13 December 2011, Boulder, CO BUFR/PrepBUFR File Processing Ruifang Li* and Ming Hu** *NCAR/MMM **Developmental Testbed Center 1 Topics Covered Basic actions and steps of BUFR file

More information

Status of OPLACE system

Status of OPLACE system Status of OPLACE system Alena Trojáková OPLACE system The common LACE observation pre-processing system (OPLACE) - aimed to support DA implementation, avoid duplication of work on observation pre-processing

More information

User s Guide Version 3.5

User s Guide Version 3.5 User s Guide Version 3.5 August 2016 Ming Hu National Oceanic and Atmospheric Administration (NOAA)/Earth System Research Laboratory Cooperative Institute for Research in Environmental Sciences (CIRES)

More information

Introduction to MODE

Introduction to MODE Introduction to MODE Verifying with Objects Presenter: Tara Jensen copyright 2017, UCAR, all rights reserved MODE Object Identification Source: Davis 2006 (Intensity presented as vertical dimension) Smoothing

More information

The challenges of the ECMWF graphics packages

The challenges of the ECMWF graphics packages The challenges of the ECMWF graphics packages Sylvie Lamy-Thépaut Graphics Section ECMWF Slide 1 1 Outline Who are we? - ECMWF - The Graphics Section What are our Missions? - Magics - Metview - Wrep :

More information

High Performance Data Efficient Interoperability for Scientific Data

High Performance Data Efficient Interoperability for Scientific Data High Performance Data Efficient Interoperability for Scientific Data Alex Ip 1, Andrew Turner 1, Dr. David Lescinsky 1 1 Geoscience Australia, Canberra, Australia Problem: Legacy Data Formats holding us

More information

SciSpark 201. Searching for MCCs

SciSpark 201. Searching for MCCs SciSpark 201 Searching for MCCs Agenda for 201: Access your SciSpark & Notebook VM (personal sandbox) Quick recap. of SciSpark Project What is Spark? SciSpark Extensions scitensor: N-dimensional arrays

More information

NCEP HPC Transition. 15 th ECMWF Workshop on the Use of HPC in Meteorology. Allan Darling. Deputy Director, NCEP Central Operations

NCEP HPC Transition. 15 th ECMWF Workshop on the Use of HPC in Meteorology. Allan Darling. Deputy Director, NCEP Central Operations NCEP HPC Transition 15 th ECMWF Workshop on the Use of HPC Allan Darling Deputy Director, NCEP Central Operations WCOSS NOAA Weather and Climate Operational Supercomputing System CURRENT OPERATIONAL CHALLENGE

More information

BIG DATA CHALLENGES A NOAA PERSPECTIVE

BIG DATA CHALLENGES A NOAA PERSPECTIVE BIG DATA CHALLENGES A NOAA PERSPECTIVE Dr. Edward J. Kearns NASA Examiner, Science and Space Branch, OMB/EOP and Chief (acting), Remote Sensing and Applications Division National Climatic Data Center National

More information

Community Tools (1) - PrepBUFR/BUFR: Basic tools, NCEP data tank, and Obsproc

Community Tools (1) - PrepBUFR/BUFR: Basic tools, NCEP data tank, and Obsproc 2011 GSI Summer Tutorial, 29 June 2011, Boulder, CO Community Tools (1) - PrepBUFR/BUFR: Basic tools, NCEP data tank, and Obsproc Ruifang Li*, Ming Hu**, and Yongrun Guo* *NCAR/MMM **Developmental Testbed

More information

Diagnostics and Exploratory Analysis Infrastructure for ACME Workflow

Diagnostics and Exploratory Analysis Infrastructure for ACME Workflow Diagnostics and Exploratory Analysis Infrastructure for ACME Workflow ORNL: Brian Smith, John Harney, Brian Jewell LLNL: Jeffrey Painter, James McEnerney, ORNL is managed by UT-Battelle for the US Department

More information

Metview 4 ECMWF s latest generation meteorological workstation

Metview 4 ECMWF s latest generation meteorological workstation from Newsletter Number 126 Winter 2010/11 COMPUTING Metview 4 ECMWF s latest generation meteorological workstation doi:10.21957/v4xszl60 This article appeared in the Computing section of ECMWF Newsletter

More information

GSI Setup, Run and Namelist

GSI Setup, Run and Namelist GSI Setup, Run and Namelist Hui Shao GSI Community Tutorial, June 28-30, 2010, Boulder, CO Observation Error Observation Observation processing and assimilation PrepBUFR and BUFR processing: 06/29, Tue

More information

Data assimilation for OpenFOAM Combining measurements and modelling

Data assimilation for OpenFOAM Combining measurements and modelling Data assimilation for OpenFOAM Combining measurements and modelling Henk Seubers, Aerospace Engineering 8 januari 2013 () Data assimilation for OpenFOAM 1 / 27 Main goal The purpose of this project is

More information

Adapting Software to NetCDF's Enhanced Data Model

Adapting Software to NetCDF's Enhanced Data Model Adapting Software to NetCDF's Enhanced Data Model Russ Rew UCAR Unidata EGU, May 2010 Overview Background What is netcdf? What is the netcdf classic data model? What is the netcdf enhanced data model?

More information

Unified Model Intercomparisons for Volcanic Ash Transport Modeling

Unified Model Intercomparisons for Volcanic Ash Transport Modeling Unified Model Intercomparisons for Volcanic Ash Transport Modeling Don Morton 1, Dèlia Arnold 2, Peter Webley 3, Gerhard Wotawa 2, Barbara Stunder 4 1 Arctic Region Supercomputing Center, University of

More information

ARCHITECTURE OF MADIS DATA PROCESSING AND DISTRIBUTION AT FSL

ARCHITECTURE OF MADIS DATA PROCESSING AND DISTRIBUTION AT FSL P2.39 ARCHITECTURE OF MADIS DATA PROCESSING AND DISTRIBUTION AT FSL 1. INTRODUCTION Chris H. MacDermaid*, Robert C. Lipschutz*, Patrick Hildreth*, Richard A. Ryan*, Amenda B. Stanley*, Michael F. Barth,

More information

Introduction of new WDCGG website. Seiji MIYAUCHI Meteorological Agency

Introduction of new WDCGG website. Seiji MIYAUCHI Meteorological Agency Introduction of new WDCGG website Seiji MIYAUCHI WDCGG@Japan Meteorological Agency 1. Introduction of new WDCGG website 2. Starting to gather and provide satellite data at WDCGG Current WDCGG website 3

More information

GEMS: Global Earth-system Monitoring using Satellite & in-situ Data

GEMS: Global Earth-system Monitoring using Satellite & in-situ Data GEMS: Global Earth-system Monitoring using Space and in-situ data Introduction to GEMS 2006 Assembly GMES Integrated Project, 12.5MEuro, 30 Institutes, 14 Countries www.ecmwf.int/research/eu_projects/gems

More information

From Integrated to Object-Oriented

From Integrated to Object-Oriented From Integrated to Object-Oriented Yannick Trémolet and Mike Fisher ECMWF 3 October 2012 Thanks to many people who have contributed to the project: Tomas Wilhelmsson, Deborah Salmond, John Hague, George

More information

CONDUIT and NAWIPS Migration to AWIPS II Status Updates Unidata Users Committee. NCEP Central Operations April 18, 2013

CONDUIT and NAWIPS Migration to AWIPS II Status Updates Unidata Users Committee. NCEP Central Operations April 18, 2013 CONDUIT and NAWIPS Migration to AWIPS II Status Updates 2013 Unidata Users Committee NCEP Central Operations April 18, 2013 CONDUIT Cooperative Opportunity for NCEP Data Using IDD Technology No changes

More information

COMMUNITY VERSION 3.4. User s Guide. July Developmental Testbed Center

COMMUNITY VERSION 3.4. User s Guide. July Developmental Testbed Center COMMUNITY VERSION 3.4 User s Guide July 2015 Developmental Testbed Center National Center for Atmospheric Research National Centers for Environmental Prediction, NOAA Global Systems Division, Earth System

More information

Converging Remote Sensing and Data Assimilation through Data Fusion

Converging Remote Sensing and Data Assimilation through Data Fusion Converging Remote Sensing and Data Assimilation through Data Fusion Benjamin T. Johnson, Ph.D. AER @ NOAA/STAR/JCSDA benjamin.t.johnson@noaa.gov The Team: Sid Boukabara Kevin Garrett Eric Maddy Ling Liu

More information

SciSpark Tutorial 101

SciSpark Tutorial 101 SciSpark Tutorial 101 Introduction to Spark Super Cool Parallel Computing In-Memory Map-Reduce It Slices, It Dices, It Minces,... So Fast, You Won t Believe It!!! ORDER NOW!!! Agenda for 101: Intro. to

More information

SCSODC: Integrating Ocean Data for Visualization Sharing and Application

SCSODC: Integrating Ocean Data for Visualization Sharing and Application IOP Conference Series: Earth and Environmental Science OPEN ACCESS SCSODC: Integrating Ocean Data for Visualization Sharing and Application To cite this article: C Xu et al 2014 IOP Conf. Ser.: Earth Environ.

More information

COMMUNITY VERSION 3.3. User s Guide. June Developmental Testbed Center

COMMUNITY VERSION 3.3. User s Guide. June Developmental Testbed Center COMMUNITY VERSION 3.3 User s Guide June 2014 Developmental Testbed Center National Center for Atmospheric Research National Centers for Environmental Prediction, NOAA Global Systems Division, Earth System

More information

User s Guide Version 3.7

User s Guide Version 3.7 User s Guide Version 3.7 November 2018 Ming Hu, Guoqing Ge National Oceanic and Atmospheric Administration (NOAA)/Earth System Research Laboratory Cooperative Institute for Research in Environmental Sciences

More information

User s Guide Version 1.3

User s Guide Version 1.3 User s Guide Version 1.3 - Compatible with GSI community release v3.7 November 2018 Hui Liu National Center for Atmospheric Research (NCAR) Ming Hu, Guoqing Ge National Oceanic and Atmospheric Administration

More information

META-T. Water Temperature Metadata Pilot Project

META-T. Water Temperature Metadata Pilot Project META-T Water Temperature Metadata Pilot Project Elanor Gowland (co-chair META-T), CLIMAR-III, 6 9 th May 2008 On behalf of Bill Burnett (co-chair META-T) Contents This presentation covers the following

More information

Gridded Data Speedwell Derived Gridded Products

Gridded Data Speedwell Derived Gridded Products Gridded Data Speedwell Derived Gridded Products Introduction Speedwell Weather offers access to a wide choice of gridded data series. These datasets are sourced from the originating agencies in their native

More information

CESM Projects Using ESMF and NUOPC Conventions

CESM Projects Using ESMF and NUOPC Conventions CESM Projects Using ESMF and NUOPC Conventions Cecelia DeLuca NOAA ESRL/University of Colorado CESM Annual Workshop June 18, 2014 Outline ESMF development update Joint CESM-ESMF projects ESMF applications:

More information

Introduction to the ClimValDiagTool

Introduction to the ClimValDiagTool Introduction to the ClimValDiagTool K. Gottschaldt & V. Eyring, 13. 2. 2013 1. General Info 2. Access miklip.dkrz.de 3. Get the code 4. Prepare data 5. Walk through an example 6. Modify the example 7.

More information

Metview Introduction

Metview Introduction Metview Introduction Fernando Ii Slide 1 Meteorological Visualisation Section ECMWF Metview - Introduction June 2013 1 Outline Introduction Interactive usage demo Slide 2 Metview - Introduction June 2013

More information

Interface, outputs, post-processing

Interface, outputs, post-processing Interface, outputs, post-processing Frédéric Pérez SMILEI training workshop November 6-7, 2017 Maison de la Simulation The minimum knowledge to use Smilei if you want to keep away from the code 1. Compile

More information

The TIDB2 Meteo Experience

The TIDB2 Meteo Experience The TIDB2 Meteo Experience Experience with the TIDB2 database interface in managing meteorological observation and forecast data João Simões ECMWF, IM (Portugal) Maria Monteiro - IM (Portugal) António

More information

HPC Input/Output. I/O and Darshan. Cristian Simarro User Support Section

HPC Input/Output. I/O and Darshan. Cristian Simarro User Support Section HPC Input/Output I/O and Darshan Cristian Simarro Cristian.Simarro@ecmwf.int User Support Section Index Lustre summary HPC I/O Different I/O methods Darshan Introduction Goals Considerations How to use

More information

User s Guide Version 1.2

User s Guide Version 1.2 User s Guide Version 1.2 - Compatible with GSI community release v3.6 September 2017 Hui Liu National Center for Atmospheric Research (NCAR) Ming Hu National Oceanic and Atmospheric Administration (NOAA)/Earth

More information