The Regional Climate Model Evalua4on System (RCMES): Introduc4on and Demonstra4on

Size: px
Start display at page:

Download "The Regional Climate Model Evalua4on System (RCMES): Introduc4on and Demonstra4on"

Transcription

1 The Regional Climate Model Evalua4on System (RCMES): Introduc4on and Demonstra4on Paul C. Loikith, Duane E. Waliser, Chris MaEmann, Jinwon Kim, Huikyo Lee, Paul M. Ramirez, Andrew F. Hart, Cameron E. Goodale, Michael J. Joyce, Shakeh E. Khudikyan, Maziyar Boustani, Kim Whitehall, Alex Goodman, Jesslyn WhiEell, and Paul Zimdars WCRP VAMOS/CORDEX Workshop on La4n- America and Caribbean CORDEX LAC Phase II- Caribbean April, 7-9, 2014 Santo Domingo, Dominican Republic

2 The Regional Climate Model Evalua4on System (RCMES) Joint collabora4on: JPL/NASA, UCLA Designed to facilitate model evalua4on and decision making Provides access to numerous (NASA) observa4on datasets (RCMED) Python- based built- in toolkit (RCMET) has regridding capabili4es and calculates and visualizes several common metrics (RMSE, Bias) Connec4on to the Earth System Grid Federa4on allowing for access to CORDEX and CMIP5 models. Ini4al target: CORDEX

3 RCMES Architecture ( Powered by Apache Open Climate Workbench) Other Data Centers (ESGF, DAAC, ExArch Network) User input Model data TRMM Metadata URL Extract OBS data Extract model data MODIS AIRS CERES Soil moisture ETC Extractor for various data formats Data Table Data Table Cloud Database Table Data Table Data Table Data Table Common Format, Na?ve grid, Efficient architecture Regridder (Put the OBS & model data on the same?me/space grid) Analyzer Calculate evalua?on metrics & assessment model input data Visualizer (Plot the metrics) Data extractor (Binary or netcdf) Use the re- gridded data for user s own analyses and VIS. Assess. modeling Raw Data: Various sources, formats, Resolu4ons, Coverage RCMED (Regional Climate Model Evalua4on Database) A large scalable database to store data from variety of sources in a common format RCMET (Regional Climate Model Evalua4on Tool) A library of code for extrac4ng data from RCMED and model and for calcula4ng evalua4on metrics

4 Evalua4on of Cloud Compu4ng for Storage & Applica4on of NASA Observa4ons Challenge Regional climate model evalua4on with daily temporal resolu4on to assess representa4on of extreme events. More voluminous, requires scalability in web services, system throughput, and also elas4city based on study demands Objec?ve Understand and evaluate popular cloud compu4ng technologies, and provide a framework for selec4ng the best one for suppor4ng Regional Climate Model Evalua4on System (RCMES) & applica4ons such as the Na4onal Climate Assessment and IPCC s CORDEX regional model evalua4ons. Results 90" 80" Conducted evalua4on demonstra4ng 44 % 70" avg query 4me speedup of PostGIS over MySQL 50" 40" for 5 years of 5 parameters of obs data in RCMES 30" Will incorporate into RCMES to facilitate NCA and CORDEX regional model evalua4ons. Second' 60" 20" 10" 0" wrm_trmm_pcp"" Query'"datapoint_id"'column'for'5'years'(2002:2007)' wrm_airs_surfairtemp_a"" ceres_par"" ceres_cre_lw_mon"" ceres_ave_sza"" MySQL"" PostgreSQL" C. MaWmann, D. Waliser, J. Kim, C. Goodale, A. Hart, P. Ramirez, D. Crichton, P. Zimdars, M. Boustani, H. Lee, P. Loikith, K. Whitehall, C. Jack, B. Hewitson. Cloud Compu4ng and Virtualiza4on Within the Regional Climate Model and Evalua4on System. Earth Science Informa0cs, 2013.

5 Meet the RCMES Team Science Team: Duane Waliser (PI, JPL/Caltech, UCLA), Jinwon Kim (UCLA), Paul Loikith (JPL/Caltech), Huikyo Lee (JPL/Caltech), Kim Whitehall (Howard University) IT Team: Chris MaWmann (PI, JPL/Caltech, UCLA), Paul Ramirez (JPL/Caltech), Cameron Goodale (JPL/Caltech), Michael Joyce (JPL/Caltech), Maziyar Boustani (JPL/Caltech), Andrew Hart (JPL/Caltech), Shakeh Khudikyan (JPL/Caltech), Jesslyn WhiEel (University of California, Berkeley), Alex Goodman (Colorado State University)

6 Ac4ve CORDEX Collabora4ons * * * * * * * *

7 Observa4ons Remote Sensing, In Situ, Reanalysis Temperature (AIRS, CRU, UDEL) Precipita4on (TRMM, CRU, UDEL, CPC, GPCP) Radia4on/clouds (CERES, MODIS) Sea surface height (AVISO) Sea surface temperature (AMSRE) Winds (QuikSCAT) Mul4variate reanalysis (MERRA, NARR, NLDAS, ERA- Interim) More to come

8 Jet Propulsion Laboratory California Ins4tute of Technology Ongoing Model Evalua4on Studies Bayesian model averaging for op4mal mul4- model ensemble configura4ons. (Huikyo Lee - lead) K- means clustering to evaluate surface temperature variance and skewness over South America (Huikyo Lee - lead). Large scale meteorological paeerns associated with temperature extremes over North America (Paul Loikith - lead )

9 Ways to Use RCMES RCMES in a virtual machine environment Downloadable from /downloads Comes with all Python libraries installed RCMES open source code from Apache Open Climate Workbench Source code downloadable from hep://climate.incubator.apache.org/ Requires all necessary Python libraries installed on local machine Easily customizable, can interact easily with your own model/obs data Available in command line and interac4ve users interface (UI) formats. Downloading instruc4ons available at

10 Future Direc4on Observa4ons and metrics con4nue to be added to RCMES Community can contribute to RCEMT via the Apache Open Climate Workbench (hep://climate. apache.org/) Integra4on of metrics into the RCMES toolkit Expanding user community and fostering new collabora4ons within CORDEX Connect to ESGF * (in progress) Webinar?

11 RCMES Demonstra4on Youtube video at: hep:///training/videos

12 Thank you! Contact informa4on: Paul Loikith: Duane Waliser: (science PI) Chris MaEmann: (IT PI) General ques4ons: rcmes- Websites: /downloads hep://climate.incubator.apache.org/ Ques4ons?

Scientific Applications of the Regional Climate Model Evaluation System (RCMES)

Scientific Applications of the Regional Climate Model Evaluation System (RCMES) Scientific Applications of the Regional Climate Model Evaluation System (RCMES) Paul C. Loikith, Duane E. Waliser, Chris Mattmann, Jinwon Kim, Huikyo Lee, Paul M. Ramirez, Andrew F. Hart, Cameron E. Goodale,

More information

CORDEX DOMAINS (plus Arctic & Antarctica)

CORDEX DOMAINS (plus Arctic & Antarctica) CORDEX DOMAINS (plus Arctic & Antarctica) 12 domains with a resolution of 0.44 (approx. 50x50km²) Focus on Africa High resolution ~0.11 x0.11 for Europe (by some institutions) The TFRCD mandate was extended

More information

CoG: The NEW ESGF WEB USER INTERFACE

CoG: The NEW ESGF WEB USER INTERFACE CoG: The NEW ESGF WEB USER INTERFACE ESGF F2F Workshop, Livermore, CA, December 2014 Luca Cinquini [1], Cecelia DeLuca [2], Sylvia Murphy [2] [1] California Ins/tute of Technology & NASA Jet Propulsion

More information

Challenges and Solutions for Future Modeling Data Analysis Systems

Challenges and Solutions for Future Modeling Data Analysis Systems Challenges and Solutions for Future Modeling Data Analysis Systems Tsengdar Lee tsengdar.j.lee@nasa.gov NASA Headquarters Dan Duffy, NASA GSFC Seungwon Lee, JPL Rama Nemani, NASA ARC Duane Waliser, JPL

More information

SciSpark 301. Build Your Own Climate Metrics

SciSpark 301. Build Your Own Climate Metrics SciSpark 301 Build Your Own Climate Metrics Agenda for 301: Access your SciSpark & Notebook VM (personal sandbox) Quick recap. of SciSpark Project PDF Clustering Use Case SciSpark Challenges & Lessons

More information

Making the Case for the ESGF and Apache: Long- Term So=ware Stewardship

Making the Case for the ESGF and Apache: Long- Term So=ware Stewardship Making the Case for the ESGF and Apache: Long- Term So=ware Stewardship Chris Ma?mann Chief Architect, Instrument and Science Data Systems, Jet Propulsion Laboratory, California Ins=tute of Technology

More information

NARCCAP: North American Regional Climate Change Assessment Program. Seth McGinnis, NCAR

NARCCAP: North American Regional Climate Change Assessment Program. Seth McGinnis, NCAR NARCCAP: North American Regional Climate Change Assessment Program Seth McGinnis, NCAR mcginnis@ucar.edu NARCCAP: North American Regional Climate Change Assessment Program Nest highresolution regional

More information

Leveraging Tools and Components from OODT and Apache within Climate Science and the Earth System Grid Federa9on

Leveraging Tools and Components from OODT and Apache within Climate Science and the Earth System Grid Federa9on Leveraging Tools and Components from OODT and Apache within Climate Science and the Earth System Grid Federa9on Luca Cinquini, Dan Crichton, Chris Ma2mann NASA Jet Propulsion Laboratory, California Ins9tute

More information

SciSpark Tutorial 101

SciSpark Tutorial 101 SciSpark Tutorial 101 Introduction to Spark Super Cool Parallel Computing In-Memory Map-Reduce It Slices, It Dices, It Minces,... So Fast, You Won t Believe It!!! ORDER NOW!!! Agenda for 101: Intro. to

More information

Collabora've Development

Collabora've Development Collabora've Development Ricardo Todling NASA GMAO 2013 Joint DTC- EMC- JCSDA GSI Workshop 1 In the Beginning NCEP: SSI (late 1980 s) Spectral formula'on of background error cov Direct assimila'on of radiances

More information

SciSpark 201. Searching for MCCs

SciSpark 201. Searching for MCCs SciSpark 201 Searching for MCCs Agenda for 201: Access your SciSpark & Notebook VM (personal sandbox) Quick recap. of SciSpark Project What is Spark? SciSpark Extensions scitensor: N-dimensional arrays

More information

Existing Solutions. Operating data services: Climate Explorer ECA&D climate4impact.eu data.knmi.nl

Existing Solutions. Operating data services: Climate Explorer ECA&D climate4impact.eu data.knmi.nl Existing Solutions Operating data services: Climate Explorer ECA&D climate4impact.eu data.knmi.nl Wim Som de Cerff, KNMI R&D Observations and Data Technology sdecerff@knmi.nl Climate data services at KNMI

More information

Gridded Data Speedwell Derived Gridded Products

Gridded Data Speedwell Derived Gridded Products Gridded Data Speedwell Derived Gridded Products Introduction Speedwell Weather offers access to a wide choice of gridded data series. These datasets are sourced from the originating agencies in their native

More information

TerraSwarm. A Machine Learning and Op0miza0on Toolkit for the Swarm. Ilge Akkaya, Shuhei Emoto, Edward A. Lee. University of California, Berkeley

TerraSwarm. A Machine Learning and Op0miza0on Toolkit for the Swarm. Ilge Akkaya, Shuhei Emoto, Edward A. Lee. University of California, Berkeley TerraSwarm A Machine Learning and Op0miza0on Toolkit for the Swarm Ilge Akkaya, Shuhei Emoto, Edward A. Lee University of California, Berkeley TerraSwarm Tools Telecon 17 November 2014 Sponsored by the

More information

Index Introduction Setting up an account Searching and accessing Download Advanced features

Index Introduction Setting up an account Searching and accessing Download Advanced features ESGF Earth System Grid Federation Tutorial Index Introduction Setting up an account Searching and accessing Download Advanced features Index Introduction IT Challenges of Climate Change Research ESGF Introduction

More information

EOSC Services & Architecture: the EOSC-hub approach Tiziana Ferrari, Project Coordinator, EGI Founda?on

EOSC Services & Architecture: the EOSC-hub approach Tiziana Ferrari, Project Coordinator, EGI Founda?on EOSC Services & Architecture: the EOSC-hub approach Tiziana Ferrari, Project Coordinator, EGI Founda?on eosc-hub.eu @EOSC_eu EOSC-hub receives funding from the European Union s Horizon 2020 research and

More information

Global Soil Wetness Project Phase 3 Atmospheric Boundary Conditions (Experiment

Global Soil Wetness Project Phase 3 Atmospheric Boundary Conditions (Experiment Global Soil Wetness Project Phase 3 Atmospheric Boundary Conditions (Experiment 1) 1. IDENTIFICATION INFORMATION Edition Version 1 Abbreviation DOI Metadata Identifier 2. CONTACT Global Soil Wetness Project

More information

ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P.

ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. Kushner, D. Waliser, S. Pascoe, A. Stephens, P. Kershaw,

More information

Observa(on Processing. Nancy Collins or

Observa(on Processing. Nancy Collins or Observa(on Processing Nancy Collins nancy@ucar.edu or dart@ucar.edu Roadmap What s in an Observa(on Provided tools and capabili(es Observa(on sources Observa(on and representa(veness error Types, Kinds,

More information

ExArch, Edinburgh, March 2014

ExArch, Edinburgh, March 2014 ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. Kushner, D. Waliser, S. Pascoe, A. Stephens, P. Kershaw,

More information

Ulrich Looser. Global Runoff Data Centre at the Federal Ins9tute of Hydrology (BfG) Koblenz, Germany

Ulrich Looser. Global Runoff Data Centre at the Federal Ins9tute of Hydrology (BfG) Koblenz, Germany TPE-GHP/GEWEX Joint Workshop Status Report of Global Data Centres GPCC (Global Precipitation Climatology Centre) HYDROLARE (International Centre on the Hydrology of Lakes and Reservoirs) GRDC (Global Runoff

More information

Distributed Online Data Access and Analysis

Distributed Online Data Access and Analysis Distributed Online Data Access and Analysis Ruixin Yang George Mason University Slides from SIESIP Partners and from NOMADS PI, Glenn K. Rutledge of US NCDC on NOMADS SIESIP: Seasonal-to-Interannual Earth

More information

Introduc)on to the RCE September 21, 2010 Len Wisniewski

Introduc)on to the RCE September 21, 2010 Len Wisniewski Introduc)on to the RCE September 21, 2010 Len Wisniewski IQSS technical services Resource support Research Compu)ng Environment (RCE): cluster compu)ng for sta)s)cal research Desktop support Computer lab

More information

Western Michigan University

Western Michigan University CS-6030 Cloud compu;ng Google App engine Sepideh Mohammadi Summer II 2017 Western Michigan University content Categories of cloud compu;ng Google cloud plaborm Google App Engine Storage technologies Datastore

More information

The CEDA Archive: Data, Services and Infrastructure

The CEDA Archive: Data, Services and Infrastructure The CEDA Archive: Data, Services and Infrastructure Kevin Marsh Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk with thanks to V. Bennett, P. Kershaw, S. Donegan and the rest of the CEDA Team

More information

Version 3 Updated: 10 March Distributed Oceanographic Match-up Service (DOMS) User Interface Design

Version 3 Updated: 10 March Distributed Oceanographic Match-up Service (DOMS) User Interface Design Distributed Oceanographic Match-up Service (DOMS) User Interface Design Shawn R. Smith 1, Jocelyn Elya 1, Adam Stallard 1, Thomas Huang 2, Vardis Tsontos 2, Benjamin Holt 2, Steven Worley 3, Zaihua Ji

More information

Data Analytics and Storage System (DASS) Mixing POSIX and Hadoop Architectures. 13 November 2016

Data Analytics and Storage System (DASS) Mixing POSIX and Hadoop Architectures. 13 November 2016 National Aeronautics and Space Administration Data Analytics and Storage System (DASS) Mixing POSIX and Hadoop Architectures 13 November 2016 Carrie Spear (carrie.e.spear@nasa.gov) HPC Architect/Contractor

More information

7 Ways to Increase Your Produc2vity with Revolu2on R Enterprise 3.0. David Smith, REvolu2on Compu2ng

7 Ways to Increase Your Produc2vity with Revolu2on R Enterprise 3.0. David Smith, REvolu2on Compu2ng 7 Ways to Increase Your Produc2vity with Revolu2on R Enterprise 3.0 David Smith, REvolu2on Compu2ng REvolu2on Compu2ng: The R Company REvolu2on R Free, high- performance binary distribu2on of R REvolu2on

More information

IS- ENES2 Key Performance Indicators

IS- ENES2 Key Performance Indicators Project Objec:ves: 1 - Foster the integra:on of the European Climate and Earth system modelling community 2 - Enhance the development of Earth System Models for the understanding of climate variability

More information

Analysis Methods in Atmospheric and Oceanic Science

Analysis Methods in Atmospheric and Oceanic Science Analysis Methods in Atmospheric and Oceanic Science AOSC 652 HDF & NetCDF files; Regression; File Compression & Data Access Week 11, Day 1 Today: Data Access for Projects; HDF & NetCDF Wed: Multiple Linear

More information

DataONE Cyberinfrastructure. Ma# Jones Dave Vieglais Bruce Wilson

DataONE Cyberinfrastructure. Ma# Jones Dave Vieglais Bruce Wilson DataONE Cyberinfrastructure Ma# Jones Dave Vieglais Bruce Wilson Foremost a Federa9on Member Nodes (MNs) Heart of the federa9on Harness the power of local cura9on Coordina9ng Nodes (CNs) Services to link

More information

System Modeling Environment

System Modeling Environment System Modeling Environment Requirements, Architecture and Implementa

More information

Objec0ves. Gain understanding of what IDA Pro is and what it can do. Expose students to the tool GUI

Objec0ves. Gain understanding of what IDA Pro is and what it can do. Expose students to the tool GUI Intro to IDA Pro 31/15 Objec0ves Gain understanding of what IDA Pro is and what it can do Expose students to the tool GUI Discuss some of the important func

More information

climate4impact.eu Christian Pagé, CERFACS

climate4impact.eu Christian Pagé, CERFACS IS-ENES2 1 st General Assembly 11-13 th June 2014 UPC Campus, Barcelona, Spain Status of infrastructure climate4impact.eu Christian Pagé, CERFACS Working teams and institutions CERFACS: Christian Pagé

More information

Tangible Visualiza.on. Andy Wu Synaesthe.c Media Lab GVU Center Georgia Ins.tute of Technology

Tangible Visualiza.on. Andy Wu Synaesthe.c Media Lab GVU Center Georgia Ins.tute of Technology Tangible Visualiza.on Andy Wu Synaesthe.c Media Lab GVU Center Georgia Ins.tute of Technology Introduc.on Informa.on Visualiza.on (Infovis) is the study of the visual representa.on of complex informa.on,

More information

First experiences of using WC(P)S at ECMWF

First experiences of using WC(P)S at ECMWF Earth Server-2 First experiences of using WC(P)S at ECMWF Julia Wagemann and Stephan Siemen European Centre for Medium-Range Weather Forecasts Workshop on Meteorological Operational Systems #OpenDataWeek

More information

InfraStructure for the European Network for Earth System modelling. From «IS-ENES» to IS-ENES2

InfraStructure for the European Network for Earth System modelling. From «IS-ENES» to IS-ENES2 InfraStructure for the European Network for Earth System modelling From «IS-ENES» to IS-ENES2 Sylvie JOUSSAUME, CNRS, Institut Pierre Simon Laplace, Coordinator ENES European Network for Earth System modelling

More information

DART Tutorial Sec'on 16: Diagnos'c Output

DART Tutorial Sec'on 16: Diagnos'c Output DART Tutorial Sec'on 16: Diagnos'c Output UCAR 214 The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons expressed

More information

DART Tutorial Sec'on 16: Diagnos'c Output

DART Tutorial Sec'on 16: Diagnos'c Output DART Tutorial Sec'on 16: Diagnos'c Output UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons expressed

More information

CCW Workshop Technical Session on Mobile Cloud Compu<ng

CCW Workshop Technical Session on Mobile Cloud Compu<ng CCW Workshop Technical Session on Mobile Cloud Compu

More information

TPP On The Cloud. Joe Slagel

TPP On The Cloud. Joe Slagel TPP On The Cloud Joe Slagel Lecture topics Introduc5on to Cloud Compu5ng and Amazon Web Services Overview of TPP Cloud components Setup trial AWS and use of the new TPP Web Launcher for Amazon (TWA) Future

More information

Introduc)on to Informa)on Visualiza)on

Introduc)on to Informa)on Visualiza)on Introduc)on to Informa)on Visualiza)on Seeing the Science with Visualiza)on Raw Data 01001101011001 11001010010101 00101010100110 11101101011011 00110010111010 Visualiza(on Applica(on Visualiza)on on

More information

CEOS Water Portal Project <<Final Project Summary>>

CEOS Water Portal Project <<Final Project Summary>> CEOS Water Portal Project Satoko Horiyama MIURA Space Applications and Operations Center (SAOC) JAXA Contents 1. Goal 2. Concept 3. History 4. Data Partners 5. Available Data

More information

Jeff Gooding Southern California Edison. Innovation at Southern California Edison

Jeff Gooding Southern California Edison. Innovation at Southern California Edison Jeff Gooding Southern California Edison Innovation at Southern California Edison Understand Cultural Risk Aversion to Innova

More information

Server-Side Processing for Large-Scale Data Analytics ESGF Compute Working Team (ESGF-CWT)

Server-Side Processing for Large-Scale Data Analytics ESGF Compute Working Team (ESGF-CWT) National Aeronautics and Space Administration Server-Side Processing for Large-Scale Data Analytics ESGF Compute Working Team (ESGF-CWT) ESGF & UV-CDAT Face-to-Face December 2014 " Daniel Duffy daniel.q.duffy@nasa.gov"

More information

How to expand the Galaxy from genes to Earth in six simple steps

How to expand the Galaxy from genes to Earth in six simple steps Department of Science and Technologies University of Naples Parthenope Mathema;cs and Computer Science Division Department of Agricultural and Biological Engineering How to expand the Galaxy from genes

More information

PyCordexer. A RegCM output format converter according to CORDEX archive specifications

PyCordexer. A RegCM output format converter according to CORDEX archive specifications PyCordexer A RegCM output format converter according to CORDEX archive specifications December 2014 2 PyCordexer The PyCordexer scripts have been developed to ease the RegCM Model User in converting variables

More information

CS 6140: Machine Learning Spring 2017

CS 6140: Machine Learning Spring 2017 CS 6140: Machine Learning Spring 2017 Instructor: Lu Wang College of Computer and Informa@on Science Northeastern University Webpage: www.ccs.neu.edu/home/luwang Email: luwang@ccs.neu.edu Logis@cs Grades

More information

AWS Iden)ty And Access Management (IAM) Manohar Rapolu

AWS Iden)ty And Access Management (IAM) Manohar Rapolu AWS Iden)ty And Access Management (IAM) Manohar Rapolu Topics Introduc5on Principals Authen5ca5on Authoriza5on Other Key Feature -> Mul5 Factor Authen5ca5on -> Rota5ng Keys -> Resolving Mul5ple Permissions

More information

Pangeo. A community-driven effort for Big Data geoscience

Pangeo. A community-driven effort for Big Data geoscience Pangeo A community-driven effort for Big Data geoscience !2 What would you like to have and why? Pangeo s vision for scientific computing in the big-data era Pangeo s Website pangeo-data.org !3 Hello!

More information

Ecography. Supplementary material

Ecography. Supplementary material Ecography ECOG-03031 Fordham, D. A., Saltré, F., Haythorne, S., Wigley, T. M. L., Otto-Bliesner, B. L., Chan, K. C. and Brooks, B. W. 2017. PaleoView: a tool for generating continuous climate projections

More information

UAB Research Compu1ng Resources and Ac1vi1es

UAB Research Compu1ng Resources and Ac1vi1es UAB Research Compu1ng Resources and Ac1vi1es Research Compu1ng Day September 13, 2012 UAB IT Research Compu1ng UAB IT Research Compu1ng Team Bob Cloud Execu1ve Director Infrastructure Services UAB IT Mike

More information

Tom Achtor, Tom Rink, Tom Whittaker. Space Science & Engineering Center (SSEC) at the University of Wisconsin - Madison

Tom Achtor, Tom Rink, Tom Whittaker. Space Science & Engineering Center (SSEC) at the University of Wisconsin - Madison Interactive Processing of Multi- and Hyper-spectral Environmental Satellite Data: The Next Generation of McIDAS EUMETSAT-AMS Conf. Amsterdam, NL 27 September 2007 Tom Achtor, Tom Rink, Tom Whittaker Space

More information

Clocker. Deploying Complex Applica3ons on Docker using Apache Brooklyn

Clocker. Deploying Complex Applica3ons on Docker using Apache Brooklyn Clocker Deploying Complex Applica3ons on Docker using Apache Brooklyn Deploying Complex Applica1ons on Docker using Apache Brooklyn Andrew Kennedy @grkvlt ApacheCon, November 2014 Budapest, Hungary Introduc1on

More information

CSE 344 Introduc/on to Data Management. Sec/on 9: AWS, Hadoop, Pig La/n Yuyin Sun

CSE 344 Introduc/on to Data Management. Sec/on 9: AWS, Hadoop, Pig La/n Yuyin Sun CSE 344 Introduc/on to Data Management Sec/on 9: AWS, Hadoop, Pig La/n Yuyin Sun Homework 8 (Last hw J) 0.5 TB (yes, TeraBytes!) of data 251 files of ~ 2GB each btc-2010-chunk-000 to btc-2010-chunk-317

More information

Improving Oceanographic Anomaly Detection Using High Performance Computing

Improving Oceanographic Anomaly Detection Using High Performance Computing Improving Oceanographic Anomaly Detection Using High Performance Computing Thomas Huang, Ed Armstrong, George Chang, Toshio Chin, Brian Wilson, Tong (Tony) Lee, Victor Zlotnicki. Jorge Vazquez and Michelle

More information

Global couplled ocean-atmosphere reanalysis and forecast with pelagic ecosystem

Global couplled ocean-atmosphere reanalysis and forecast with pelagic ecosystem Global couplled oceanatmosphere reanalysis and forecast with pelagic ecosystem 1. IDENTIFICATION INFORMATION Abbreviation Metadata Identifier Global couplled ocean-atmosphere reanalysis and forecast with

More information

Introduction to the ESMValTool

Introduction to the ESMValTool Introduction to the ESMValTool 1. General Info 2. Installation 3. Selecting data and diagnostics 4. Recent developments in EMBRACE 5. Modify plots 6. Options to contribute your own diagnostics 7. How to

More information

REsources linkage for E-scIence - RENKEI -

REsources linkage for E-scIence - RENKEI - REsources linkage for E-scIence - - hlp://www.e- sciren.org/ REsources linkage for E- science () is a research and development project for new middleware technologies to enable e- science communi?es. ""

More information

Levels of Service Authors: R. Duerr, A. Leon, D. Miller, D.J Scott Date 7/31/2009

Levels of Service Authors: R. Duerr, A. Leon, D. Miller, D.J Scott Date 7/31/2009 Levels of Service Authors: R. Duerr, A. Leon, D. Miller, D.J Scott Date 7/31/2009 CHANGE LOG Revision Date Description Author 1.0 7/31/2009 Original draft Duerr, Leon, Miller, Scott 2.0 3/19/2010 Updated

More information

TerraSwarm. A Machine Learning and Op0miza0on Toolkit for the Swarm. Ilge Akkaya, Shuhei Emoto, Edward A. Lee. University of California, Berkeley

TerraSwarm. A Machine Learning and Op0miza0on Toolkit for the Swarm. Ilge Akkaya, Shuhei Emoto, Edward A. Lee. University of California, Berkeley TerraSwarm A Machine Learning and Op0miza0on Toolkit for the Swarm Ilge Akkaya, Shuhei Emoto, Edward A. Lee University of California, Berkeley TerraSwarm Tools Telecon 17 November 2014 Sponsored by the

More information

Collaborative data-driven science. Collaborative data-driven science

Collaborative data-driven science. Collaborative data-driven science Alex Szalay ! Started with the SDSS SkyServer! Built very quickly in 2001! Goal: instant access to rich content! Idea: bring the analysis to the data! Interac

More information

Register Alloca.on Deconstructed. David Ryan Koes Seth Copen Goldstein

Register Alloca.on Deconstructed. David Ryan Koes Seth Copen Goldstein Register Alloca.on Deconstructed David Ryan Koes Seth Copen Goldstein 12th Interna+onal Workshop on So3ware and Compilers for Embedded Systems April 24, 12009 Register Alloca:on Problem unbounded number

More information

Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners

Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners 24th Forum ORAP Cite Scientifique; Lille, France March 26, 2009 Don Middleton National

More information

Internet2 Webinar: Confluence BoF. April 28, 2009

Internet2 Webinar: Confluence BoF. April 28, 2009 Internet2 Webinar: Confluence BoF April 28, 2009 Ques=ons to answer How massively can Confluence scale? What are its limita=ons? How does clustering help Confluence scale? What are some guidelines in tuning

More information

UC3: A Framework for Coopera;ve Compu;ng at the University of Chicago

UC3: A Framework for Coopera;ve Compu;ng at the University of Chicago UC3: A Framework for Coopera;ve Compu;ng at the University of Chicago Marco Mambelli (marco@hep.uchicago.edu), Rob Gardner Computa;on and Enrico Fermi Ins;tutes UC Compu;ng Coopera;ve A shared Campus distributed

More information

Web Interface to Materials Simulations

Web Interface to Materials Simulations Web Interface to Materials Simulations Web Interface Generator and Legacy Application Façade Portal Development Team Funding Akos J. Czikmantory (JPL - Wiglaf) DARPA-PROM Hook Hua (JPL - Wiglaf ) JPL SRRF

More information

Matlab stoqstoolbox for accessing in situ measurements from STOQS

Matlab stoqstoolbox for accessing in situ measurements from STOQS Matlab stoqstoolbox for accessing in situ measurements from STOQS MBARI Summer Internship Project 2012 Francisco Lopez Castejon (Mentor: Mike McCann) INDEX Abstract... 4 Introduction... 5 In situ measurement

More information

Energy- Aware Time Change Detec4on Using Synthe4c Aperture Radar On High- Performance Heterogeneous Architectures: A DDDAS Approach

Energy- Aware Time Change Detec4on Using Synthe4c Aperture Radar On High- Performance Heterogeneous Architectures: A DDDAS Approach Energy- Aware Time Change Detec4on Using Synthe4c Aperture Radar On High- Performance Heterogeneous Architectures: A DDDAS Approach Sanjay Ranka (PI) Sartaj Sahni (Co- PI) Mark Schmalz (Co- PI) University

More information

Sea Level CCI project Phase II. System Engineering Status

Sea Level CCI project Phase II. System Engineering Status ESA Climate Change Initiative Sea Level CCI project Phase II System Engineering Status WP3000 System Evolution Tasks WP3100: System Engineering activities (CGI-led) o WP3110: System Specification (CGI)

More information

Metadata Zoo Dataset Metadata Rebecca Koskela Execu4ve Director, DataONE

Metadata Zoo Dataset Metadata Rebecca Koskela Execu4ve Director, DataONE Metadata Zoo Dataset Metadata Rebecca Koskela Execu4ve Director, DataONE eurocris September 9, 2013 Outline Data Challenges Metadata Solu=on DataONE addressing the Data Challenge Enabling Scien=fic Discovery

More information

Submitted to: Dr. Sunnie Chung. Presented by: Sonal Deshmukh Jay Upadhyay

Submitted to: Dr. Sunnie Chung. Presented by: Sonal Deshmukh Jay Upadhyay Submitted to: Dr. Sunnie Chung Presented by: Sonal Deshmukh Jay Upadhyay Submitted to: Dr. Sunny Chung Presented by: Sonal Deshmukh Jay Upadhyay What is Apache Survey shows huge popularity spike for Apache

More information

McIDAS-V - A powerful data analysis and visualization tool for multi and hyperspectral environmental satellite data *

McIDAS-V - A powerful data analysis and visualization tool for multi and hyperspectral environmental satellite data * McIDAS-V - A powerful data analysis and visualization tool for multi and hyperspectral environmental satellite data * Thomas Achtor, Thomas Rink, Thomas Whittaker, David Parker and David Santek Space Science

More information

UAB IT Research Compu3ng Update

UAB IT Research Compu3ng Update UAB IT Research Compu3ng Update UAB IT Research Compu3ng Day September 15,2011 Discussion Topics Research Applica3ons Developer Tools Hardware Networks People Current Projects Projected Growth in Data

More information

CMhyd User Manual. Documentation for preparing simulated climate change data for hydrologic impact studies

CMhyd User Manual. Documentation for preparing simulated climate change data for hydrologic impact studies Documentation for preparing simulated climate change data for hydrologic impact studies May, 2016 Hendrik Rathjens, Katrin Bieger, Raghavan Srinivasan, Indrajeet Chaubey, Jeffrey G. Arnold In association

More information

Desarrollo de una herramienta de visualización de datos oceanográficos: Modelos y Observaciones

Desarrollo de una herramienta de visualización de datos oceanográficos: Modelos y Observaciones Desarrollo de una herramienta de visualización de datos oceanográficos: Modelos y Observaciones J. Fernandez, J. Lopez, I. Carlos, F. Jerez, F. Hermosilla, M. Espino SIMO, LIM-UPC, CIIRC FIELD_AC 7 th

More information

PyTables. An on- disk binary data container, query engine and computa:onal kernel. Francesc Alted

PyTables. An on- disk binary data container, query engine and computa:onal kernel. Francesc Alted PyTables An on- disk binary data container, query engine and computa:onal kernel Francesc Alted Tutorial for the PyData Conference, October 2012, New York City 10 th anniversary of PyTables Hi!, PyTables

More information

Data Access and Analysis with Distributed, Federated Data Servers in climateprediction.net

Data Access and Analysis with Distributed, Federated Data Servers in climateprediction.net Data Access and Analysis with Distributed, Federated Data Servers in climateprediction.net Neil Massey 1 neil.massey@comlab.ox.ac.uk Tolu Aina 2, Myles Allen 2, Carl Christensen 1, David Frame 2, Daniel

More information

Cross- Valida+on & ROC curve. Anna Helena Reali Costa PCS 5024

Cross- Valida+on & ROC curve. Anna Helena Reali Costa PCS 5024 Cross- Valida+on & ROC curve Anna Helena Reali Costa PCS 5024 Resampling Methods Involve repeatedly drawing samples from a training set and refibng a model on each sample. Used in model assessment (evalua+ng

More information

Stay Informed During and AEer OpenWorld

Stay Informed During and AEer OpenWorld Stay Informed During and AEer OpenWorld TwiIer: @OracleBigData, @OracleExadata, @Infrastructure Follow #CloudReady LinkedIn: Oracle IT Infrastructure Oracle Showcase Page Oracle Big Data Oracle Showcase

More information

Grenville Whelan High- Availability.com. Copyright, High- Availability.Com Limited, 2014

Grenville Whelan High- Availability.com. Copyright, High- Availability.Com Limited, 2014 Grenville Whelan High- vailability.com High- vailability.com Spun- off from Sun/Solaris onsulng Pracce in 995 RSF- first independent commercial H soluon for UNIX Heritage in enterprise SPR/Solaris/Oracle

More information

Privacy-Preserving Shortest Path Computa6on

Privacy-Preserving Shortest Path Computa6on Privacy-Preserving Shortest Path Computa6on David J. Wu, Joe Zimmerman, Jérémy Planul, and John C. Mitchell Stanford University Naviga6on desired des@na@on current posi@on Naviga6on: A Solved Problem?

More information

Cloud Data Management System (CDMS)

Cloud Data Management System (CDMS) Cloud Management System (CMS) Wiqar Chaudry Solu9ons Engineer Senior Advisor CMS Overview he OpenStack cloud data management system features a canonical data modeling framework designed to broker context

More information

International Journal of Informative & Futuristic Research ISSN:

International Journal of Informative & Futuristic Research ISSN: Research Paper Volume 3 Issue 9 May 2016 International Journal of Informative & Futuristic Research Development Of An Algorithm To Data Retrieval And Data Conversion For The Analysis Of Spatiotemporal

More information

Python: Working with Multidimensional Scientific Data. Nawajish Noman Deng Ding

Python: Working with Multidimensional Scientific Data. Nawajish Noman Deng Ding Python: Working with Multidimensional Scientific Data Nawajish Noman Deng Ding Outline Scientific Multidimensional Data Ingest and Data Management Analysis and Visualization Extending Analytical Capabilities

More information

Ensemble- Based Characteriza4on of Uncertain Features Dennis McLaughlin, Rafal Wojcik

Ensemble- Based Characteriza4on of Uncertain Features Dennis McLaughlin, Rafal Wojcik Ensemble- Based Characteriza4on of Uncertain Features Dennis McLaughlin, Rafal Wojcik Hydrology TRMM TMI/PR satellite rainfall Neuroscience - - MRI Medicine - - CAT Geophysics Seismic Material tes4ng Laser

More information

Oracle Applica7on Express (APEX) For E- Business Suite Repor7ng. Your friend in the business.

Oracle Applica7on Express (APEX) For E- Business Suite Repor7ng. Your friend in the business. Oracle Applica7on Express (APEX) For E- Business Suite Repor7ng Your friend in the business. 1 Presenter Jamie Stokes Senior Director Oracle Technology Services Email: jstokes@smartdogservices.com LinkedIn:

More information

Using the Network Common Data Form for storage of atmospheric data

Using the Network Common Data Form for storage of atmospheric data Using the Network Common Data Form for storage of atmospheric data Maarten Plieger Royal Netherlands Meteorological Institute The NetCDF4 data format (Network Common Data Form) is used within the ADAGUC

More information

Leveraging User Session Data to Support Web Applica8on Tes8ng

Leveraging User Session Data to Support Web Applica8on Tes8ng Leveraging User Session Data to Support Web Applica8on Tes8ng Authors: Sebas8an Elbaum, Gregg Rotheermal, Srikanth Karre, and Marc Fisher II Presented By: Rajiv Jain Outline Introduc8on Related Work Tes8ng

More information

Hobbes: Composi,on and Virtualiza,on as the Founda,ons of an Extreme- Scale OS/R

Hobbes: Composi,on and Virtualiza,on as the Founda,ons of an Extreme- Scale OS/R Hobbes: Composi,on and Virtualiza,on as the Founda,ons of an Extreme- Scale OS/R Ron Brightwell, Ron Oldfield Sandia Na,onal Laboratories Arthur B. Maccabe, David E. Bernholdt Oak Ridge Na,onal Laboratory

More information

Distributed Systems INF Michael Welzl

Distributed Systems INF Michael Welzl Distributed Systems INF 3190 Michael Welzl What is a distributed system (DS)? Many defini8ons [Coulouris & Emmerich] A distributed system consists of hardware and sodware components located in a network

More information

Data Reference Syntax Governing Standards within Climate Research Data archived in the ESGF

Data Reference Syntax Governing Standards within Climate Research Data archived in the ESGF Data Reference Syntax Governing Standards within Climate Research Data archived in the ESGF Michael Kolax Swedish Meteorological and Hydrological Institute Motivation for a DRS within CMIP5 In CMIP5 the

More information

New Datasets, Functionality and Future Development. Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama(

New Datasets, Functionality and Future Development. Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama( HYCOM Data Service New Datasets, Functionality and Future Development Ashwanth Srinivasan, (FSU) Steve Hankin (NOAA/PMEL) Major contributors: Jon Callahan (Mazama( Consulting) Roland Schweitzer (Weathertop

More information

Scalable Software Components for Ultrascale Visualization Applications

Scalable Software Components for Ultrascale Visualization Applications Scalable Software Components for Ultrascale Visualization Applications Wes Kendall, Tom Peterka, Jian Huang SC Ultrascale Visualization Workshop 2010 11-15-2010 Primary Collaborators Jian Huang Tom Peterka

More information

Introduc)on to High Performance Compu)ng Advanced Research Computing

Introduc)on to High Performance Compu)ng Advanced Research Computing Introduc)on to High Performance Compu)ng Advanced Research Computing Outline What cons)tutes high performance compu)ng (HPC)? When to consider HPC resources What kind of problems are typically solved?

More information

Simulation-time data analysis and I/O acceleration at extreme scale with GLEAN

Simulation-time data analysis and I/O acceleration at extreme scale with GLEAN Simulation-time data analysis and I/O acceleration at extreme scale with GLEAN Venkatram Vishwanath, Mark Hereld and Michael E. Papka Argonne Na

More information

Data Portal and Integra.on in JAMSTEC

Data Portal and Integra.on in JAMSTEC Data Portal and Integra.on in JAMSTEC Yasunori Hanafusa Data Research Center for Marine-Earth Sciences (DrC) Agency for Marine-Earth Science and Technology (JAMSTEC) 1 Overview of Data Management in JAMSTEC

More information

Systems Engineering Capabili2es

Systems Engineering Capabili2es Systems Engineering Capabili2es Purdue University November 9, 2010 Integrated Deepwater System Concept US Coast Guard / ICGS Recent History of SE at Purdue 2003 Purdue College of Engineering ini2ates Signature

More information

Satellite Constella,on Mission Design using Model- Based Systems Engineering and Observing System Simula,on Experiments

Satellite Constella,on Mission Design using Model- Based Systems Engineering and Observing System Simula,on Experiments Satellite Constella,on Mission Design using Model- Based Systems Engineering and Observing System Simula,on Experiments Sreeja Nag PhD Candidate in Aerospace Engineering Massachusetts Institute of Technology

More information

International Big Science Coming to Your Campus Soon (Sooner Than You Think )

International Big Science Coming to Your Campus Soon (Sooner Than You Think ) International Big Science Coming to Your Campus Soon (Sooner Than You Think ) Lauren Rotman ESnet Science Engagement Group Lead April 7, 2014 ESnet Supports DOE Office of Science Office of Science provides

More information