Kepler Scientific Workflow and Climate Modeling
|
|
- Madeline Atkinson
- 5 years ago
- Views:
Transcription
1 Kepler Scientific Workflow and Climate Modeling Ufuk Turuncoglu Istanbul Technical University Informatics Institute Cecelia DeLuca Sylvia Murphy NOAA/ESRL Computational Science and Engineering Dept. NESII 03/06/2014, IS-ENES Workshop on Workflows in Earth System Modeling
2 Outline Motivation Our Approach CCSM Workflow Components Provenance and Metadata Collection Use Case Basic CCSM configuration (all components are active) Computing Platform: TeraGrid and Cluster (OTP) Experiences Plans
3 Common Problems in Earth System Modeling Changing the science in complex ESMs Shift from standalone model components (atm, ocn, lnd etc.) to multi-component ESMs (high resolution in time and space) Hard to use modeling systems (build and run in numerous kind of computational resources) Increased volume of data (hard to transfer, store and process) Increasing needs to test the models (especially in development stage) regularly using standardized test cases to find bugs, check the performance and test the integrity of the modeling system Increasing needs to reproduce the results (in different computational platform) Involve numerous parameter changes that are hard to record and track (ensemble runs, model sensitivity
4 Common Problems in Earth System Modeling Changing the science in complex HPCs Involve many technologies each with its own learning curve (operating systems, job schedulers, parallel file systems, programming and usage of GPU systems) It is hard and time consuming to develop and test the ESMs in HPC systems (different authentication systems, data transfer protocols and software variants compilers, libraries etc.)
5 Proposed Approach Integrating ESMs with Scientific Workflow (SWF) Applications The SWF application can act as an abstraction layers (hides details of the different technologies computing environment) It could orchestrate underlying ESMs, pre/postprocessing tools and data repositories It could collect metadata and provenance information in a standardized and automated way It creates a work environment that facilitates easy and standardized information exchange in a efficient way Prototype System (collaboration with NOAA, ESMF Team) Create self-describing earth system model (NCAR s CESM)
6 Components of Prototype Application The prototype system is tested in both grid environment (TeraGrid) and conventional cluster (NCAR s Bluefire - SSO)
7 Kepler WF Kepler WF and Modifications Open source, platform independent (Java), supports different model of computations (MoC) using directors, modular (new modules can be designed using Java) Modifications: new module earth Grid based Actors CESM Actors Post-processing Actors
8 CCSM4 (= CESM) is a multi-component global ESM and which makes it complicated to collect provenance and metadata information. The main output format is XML Provenance / Metadata Collection Tool Hierarchical (Multi-Layered) Approach
9 Conceptual CCSM Workflow
10 Kepler CCSM Workflow: Teragrid build model collect provenance info creates WS-GRAM XML job description files automatically run model
11 Kepler CCSM Workflow: Cluster (OTP) build model run model collect provenance info workflow wide global parameters Modified WF from TeraGrid to
12 Examples of Collected Provenance Information System Layer: Information related with OS Perl OS type, patch version and level, kernel This script also responsible to define environment variables for the commands that are triggered by workflow.
13 Examples of Collected Provenance Information Application Layer: ESMF Attribute Class Information related with exchange fields. It can be integrated in model (atm, ocn, ) and/or driver level
14 Examples of Collected Provenance Information Application Layer: Information related to build stage A Python script Compiler type and version, used compiler flags, list of environment variables and their values
15 Follow-up Workflows The follow-up workflow is triggered by another workflow Modified Kepler loader Job description in XML form
16 Lesson Learned in Prototype WF Design Was Kepler scientific WF a good choice? Easy to develop new actors (Java - Object Oriented Design) Different director options (sequential, parallel) The provenance module is not strong There is no any mechanisms to keep track workflow versions (SVN, Git type approach needed) The checkpoint /restart mechanism? Integrating model components with WFs is not easy Models have non-standardized interfaces (i.e. ASCII formatted configuration files or namelists) (CESM uses XML ) The current approach needs to develop model specific actors to configure, build and run the model It is not a generic solution and sensitive to change in
17 Lesson Learned in Prototype WF Design WFs and integration with computing platforms Job submission, management and monitoring via various type of job scheduler (LSF, OpenPBS etc.) and different authentication systems (OTP, VPN etc.) A set of global environment variables (TG_CLUSTER_HOME, TG_CLUSTER_PFS etc.) that point user scratch, temp and archive directories might help to create generic workflows Controlling whole environment (model, computing system, processing and visualization tools) from the external WF systems are not efficient (Web Services - client / server?) In general, WF needs continuous interaction with computing platform (monitoring, error checking etc.) Creating end-to-end WF system for ESS is a giant task. It needs more collaboration!
18 What might be needed in the future? We need to have a new approach (just thoughts ) Integrating models with standardized web services (WS) and hooking them with the WS actors in workflow applications The client can be simplified by this way. Don t need to create custom actors for all model and external tools Kepler has a set of WS actors. They can be used to interact with SOAP services The ESMF web services extension (ESMF-WS) and OpenMI might help to create web services enables model components
19 What might be needed in the future? We need to have a new approach (just thoughts ) Design of Web-based WF client (HTML5, Draw2D, )? It can be used as a centralized work and monitoring environment. In-situ visualization approach can be an option to monitor ongoing simulations (i.e. VTK based Catalyst ) A new provenance collection tool / mechanism might be designed using guideline of Metafor CIM Along with the help of ontologies - RDF, OWL It can be tightly integrated to the used ESS (ESMF Attribute type approach) Interaction with ESGF type data portals are crucial (model inputs, data processing and visualization workflows etc.)
20 More Information
21 Questions! Contact: Ufuk Utku Turunçoğlu
Introduction to Regional Earth System Model (RegESM)
Introduction to Regional Earth System Model (RegESM) Ufuk Turuncoglu Istanbul Technical University Informatics Institute 14/05/2014, 7th ICTP Workshop on the Theory and Use of Regional Climate Models Outline
More informationRegCM-ROMS Tutorial: Coupling RegCM-ROMS
RegCM-ROMS Tutorial: Coupling RegCM-ROMS Ufuk Utku Turuncoglu ICTP (International Center for Theoretical Physics) Earth System Physics Section - Outline Outline Information about coupling and ESMF Installation
More informationEuropean and international background
IS-ENES2 General Assembly 11-13th June 2014 Barcelona European and international background Metadata and ES-DOC (including statistical downscaling) Sébastien Denvil, Mark Greenslade, Allyn Treshansky,
More informationThe Earth System Modeling Framework (and Beyond)
The Earth System Modeling Framework (and Beyond) Fei Liu NOAA Environmental Software Infrastructure and Interoperability http://www.esrl.noaa.gov/nesii/ March 27, 2013 GEOSS Community ESMF is an established
More informationBuilding a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners
Building a Global Data Federation for Climate Change Science The Earth System Grid (ESG) and International Partners 24th Forum ORAP Cite Scientifique; Lille, France March 26, 2009 Don Middleton National
More informationCESM2 Software Update. Mariana Vertenstein CESM Software Engineering Group
CESM2 Software Update Mariana Vertenstein CESM Software Engineering Group Outline CMIP6 Computational Performance Cheyenne Status CESM2 new user-friendly infrastructure features CIME New porting capabilities
More informationRegCM-ROMS Tutorial: Introduction to ROMS Ocean Model
RegCM-ROMS Tutorial: Introduction to ROMS Ocean Model Ufuk Utku Turuncoglu ICTP (International Center for Theoretical Physics) Earth System Physics Section - Outline Outline Introduction Grid generation
More informationEnd-to-end optimization potentials in HPC applications for NWP and Climate Research
End-to-end optimization potentials in HPC applications for NWP and Climate Research Luis Kornblueh and Many Colleagues and DKRZ MAX-PLANCK-GESELLSCHAFT ... or a guided tour through the jungle... MAX-PLANCK-GESELLSCHAFT
More informationContinuous integration & continuous delivery. COSC345 Software Engineering
Continuous integration & continuous delivery COSC345 Software Engineering Outline Integrating different teams work, e.g., using git Defining continuous integration / continuous delivery We use continuous
More informationA Software Developing Environment for Earth System Modeling. Depei Qian Beihang University CScADS Workshop, Snowbird, Utah June 27, 2012
A Software Developing Environment for Earth System Modeling Depei Qian Beihang University CScADS Workshop, Snowbird, Utah June 27, 2012 1 Outline Motivation Purpose and Significance Research Contents Technology
More informationIndex Introduction Setting up an account Searching and accessing Download Advanced features
ESGF Earth System Grid Federation Tutorial Index Introduction Setting up an account Searching and accessing Download Advanced features Index Introduction IT Challenges of Climate Change Research ESGF Introduction
More informationScientific Workflow Tools. Daniel Crawl and Ilkay Altintas San Diego Supercomputer Center UC San Diego
Scientific Workflow Tools Daniel Crawl and Ilkay Altintas San Diego Supercomputer Center UC San Diego 1 escience Today Increasing number of Cyberinfrastructure (CI) technologies Data Repositories: Network
More informationAccelerating the Scientific Exploration Process with Kepler Scientific Workflow System
Accelerating the Scientific Exploration Process with Kepler Scientific Workflow System Jianwu Wang, Ilkay Altintas Scientific Workflow Automation Technologies Lab SDSC, UCSD project.org UCGrid Summit,
More information2/12/11. Addendum (different syntax, similar ideas): XML, JSON, Motivation: Why Scientific Workflows? Scientific Workflows
Addendum (different syntax, similar ideas): XML, JSON, Python (a) Python (b) w/ dickonaries XML (a): "meta schema" JSON syntax LISP Source: h:p://en.wikipedia.org/wiki/json XML (b): "direct" schema Source:
More informationGrid-Based Data Mining and the KNOWLEDGE GRID Framework
Grid-Based Data Mining and the KNOWLEDGE GRID Framework DOMENICO TALIA (joint work with M. Cannataro, A. Congiusta, P. Trunfio) DEIS University of Calabria ITALY talia@deis.unical.it Minneapolis, September
More informationCESM Projects Using ESMF and NUOPC Conventions
CESM Projects Using ESMF and NUOPC Conventions Cecelia DeLuca NOAA ESRL/University of Colorado CESM Annual Workshop June 18, 2014 Outline ESMF development update Joint CESM-ESMF projects ESMF applications:
More informationGetting up and running with CESM Cécile Hannay Climate and Global Dynamics (CGD), NCAR
Getting up and running with CESM Cécile Hannay Climate and Global Dynamics (CGD), NCAR NCAR is sponsored by the National Science Foundation Why CESM? State of the Art Climate Model Widely used by the Climate
More informationChapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT.
Chapter 4:- Introduction to Grid and its Evolution Prepared By:- Assistant Professor SVBIT. Overview Background: What is the Grid? Related technologies Grid applications Communities Grid Tools Case Studies
More informationCloud Computing. Up until now
Cloud Computing Lecture 4 and 5 Grid: 2012-2013 Introduction. Up until now Definition of Cloud Computing. Grid Computing: Schedulers: Condor SGE 1 Summary Core Grid: Toolkit Condor-G Grid: Conceptual Architecture
More informationProvenance Manager: PROV-man an Implementation of the PROV Standard. Ammar Benabadelkader Provenance Taskforce Budapest, 24 March 2014
Provenance Manager: PROV-man an Implementation of the PROV Standard Ammar Benabadelkader Provenance Taskforce Budapest, 24 March 2014 Outlines Motivation State-of-the-art PROV-man The Approach, the data
More informationCommon Infrastructure for Modeling Earth (CIME) and MOM6. Mariana Vertenstein CESM Software Engineering Group
Common Infrastructure for Modeling Earth (CIME) and MOM6 Mariana Vertenstein CESM Software Engineering Group Outline What is CIME? New CIME coupling infrastructure and MOM6 CESM2/DART Data Assimilation
More informationBelle II - Git migration
Belle II - Git migration Why git? Stash GIT service managed by DESY Powerful branching and merging capabilities Resolution of (JIRA) issues directly be map to branches and commits Feature freeze in pre-release
More informationHands-on tutorial on usage the Kepler Scientific Workflow System
Hands-on tutorial on usage the Kepler Scientific Workflow System (including INDIGO-DataCloud extension) RIA-653549 Michał Konrad Owsiak (@mkowsiak) Poznan Supercomputing and Networking Center michal.owsiak@man.poznan.pl
More informationDigital Curation and Preservation: Defining the Research Agenda for the Next Decade
Storage Resource Broker Digital Curation and Preservation: Defining the Research Agenda for the Next Decade Reagan W. Moore moore@sdsc.edu http://www.sdsc.edu/srb Background NARA research prototype persistent
More informationKnowledge Discovery Services and Tools on Grids
Knowledge Discovery Services and Tools on Grids DOMENICO TALIA DEIS University of Calabria ITALY talia@deis.unical.it Symposium ISMIS 2003, Maebashi City, Japan, Oct. 29, 2003 OUTLINE Introduction Grid
More informationKepler and Grid Systems -- Early Efforts --
Distributed Computing in Kepler Lead, Scientific Workflow Automation Technologies Laboratory San Diego Supercomputer Center, (Joint work with Matthew Jones) 6th Biennial Ptolemy Miniconference Berkeley,
More informationUniform Resource Locator Wide Area Network World Climate Research Programme Coupled Model Intercomparison
Glossary API Application Programming Interface AR5 IPCC Assessment Report 4 ASCII American Standard Code for Information Interchange BUFR Binary Universal Form for the Representation of meteorological
More informationCSinParallel Workshop. OnRamp: An Interactive Learning Portal for Parallel Computing Environments
CSinParallel Workshop : An Interactive Learning for Parallel Computing Environments Samantha Foley ssfoley@cs.uwlax.edu http://cs.uwlax.edu/~ssfoley Josh Hursey jjhursey@cs.uwlax.edu http://cs.uwlax.edu/~jjhursey/
More informationCoG: The NEW ESGF WEB USER INTERFACE
CoG: The NEW ESGF WEB USER INTERFACE ESGF F2F Workshop, Livermore, CA, December 2014 Luca Cinquini [1], Cecelia DeLuca [2], Sylvia Murphy [2] [1] California Ins/tute of Technology & NASA Jet Propulsion
More informationThe CIME Case Control System
The CIME Case Control System An Object Oriented Python Data Driven Workflow Control System for Earth System Models Jim Edwards 22 nd Annual Community Earth System Model Workshop Boulder, CO 19-22 June
More informationContinuous Integration (CI) with Jenkins
TDDC88 Lab 5 Continuous Integration (CI) with Jenkins This lab will give you some handson experience in using continuous integration tools to automate the integration periodically and/or when members of
More informationScience-as-a-Service
Science-as-a-Service The iplant Foundation Rion Dooley Edwin Skidmore Dan Stanzione Steve Terry Matthew Vaughn Outline Why, why, why! When duct tape isn t enough Building an API for the web Core services
More informationNCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017
NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017 Overview The Globally Accessible Data Environment (GLADE) provides centralized file storage for HPC computational, data-analysis,
More informationHow to build Scientific Gateways with Vine Toolkit and Liferay/GridSphere framework
How to build Scientific Gateways with Vine Toolkit and Liferay/GridSphere framework Piotr Dziubecki, Piotr Grabowski, Michał Krysiński, Tomasz Kuczyński, Dawid Szejnfeld, Dominik Tarnawczyk, Gosia Wolniewicz
More informationIntroduction of NAREGI-PSE implementation of ACS and Replication feature
Introduction of NAREGI-PSE implementation of ACS and Replication feature January. 2007 NAREGI-PSE Group National Institute of Informatics Fujitsu Limited Utsunomiya University National Research Grid Initiative
More informationScientific Workflows
Scientific Workflows Overview More background on workflows Kepler Details Example Scientific Workflows Other Workflow Systems 2 Recap from last time Background: What is a scientific workflow? Goals: automate
More informationInfraStructure for the European Network for Earth System modelling. From «IS-ENES» to IS-ENES2
InfraStructure for the European Network for Earth System modelling From «IS-ENES» to IS-ENES2 Sylvie JOUSSAUME, CNRS, Institut Pierre Simon Laplace, Coordinator ENES European Network for Earth System modelling
More informationWhere we are so far. Intro to Data Integration (Datalog, mediators, ) more to come (your projects!): schema matching, simple query rewriting
Where we are so far Intro to Data Integration (Datalog, mediators, ) more to come (your projects!): schema matching, simple query rewriting Intro to Knowledge Representation & Ontologies description logic,
More informationRemote Workflow Enactment using Docker and the Generic Execution Framework in EUDAT
Remote Workflow Enactment using Docker and the Generic Execution Framework in EUDAT Asela Rajapakse Max Planck Institute for Meteorology EUDAT receives funding from the European Union's Horizon 2020 programme
More informationCESM Workflow Refactor Project Land Model and Biogeochemistry Working Groups 2015 Winter Meeting CSEG & ASAP/CISL
CESM Workflow Refactor Project Land Model and Biogeochemistry Working Groups 2015 Winter Meeting Alice Bertini Sheri Mickelson CSEG & ASAP/CISL CESM Workflow Refactor Project Who s involved? Joint project
More informationRegular Forum of Lreis. Speechmaker: Gao Ang
Regular Forum of Lreis Speechmaker: Gao Ang Content: A. Overview of Eclipse Project B. Rich Client Platform C. The progress of ustudio Project D. The development of Grid technology and Grid GIS E. Future
More informationESMF. Earth System Modeling Framework. Carsten Lemmen. Schnakenbek, 17 Sep /23
1/23 ESMF Earth System Modeling Framework Carsten Lemmen Schnakenbek, 17 Sep 2013 2/23 Why couple? GEOS5 vorticity We live in a coupled world combine, extend existing models (domains + processes) reuse
More information- C3Grid Stephan Kindermann, DKRZ. Martina Stockhause, MPI-M C3-Team
A Collaborative Environment for Climate Data Handling - Stephan Kindermann, DKRZ Martina Stockhause, MPI-M C3-Team 10.06. 2008 Motivation Model Output Data + Observation Data + TeraByte Analysis Data Expected
More informationACME Exploratory Analysis and Classic Diagnostics Viewer
ACME Exploratory Analysis and Classic Diagnostics Viewer Raymond Borges Marcia Branstetter Katherine Evans John Harney Brian Jewell Benjamin Mayer Jeff Painter Galen Shipman Brian Smith Chad Steed Dean
More informationScientific Software Development with Eclipse
Scientific Software Development with Eclipse A Best Practices for HPC Developers Webinar Gregory R. Watson ORNL is managed by UT-Battelle for the US Department of Energy Contents Downloading and Installing
More informationApplication Development and Deployment With MATLAB
Application Development and Deployment With Jean-Philippe Villaréal Application Engineer Applications Engineering Group MathWorks Benelux June 11, 2015 2015 The MathWorks, Inc. 1 Typical Industry Challenges
More informationSan Diego Supercomputer Center, UCSD, U.S.A. The Consortium for Conservation Medicine, Wildlife Trust, U.S.A.
Accelerating Parameter Sweep Workflows by Utilizing i Ad-hoc Network Computing Resources: an Ecological Example Jianwu Wang 1, Ilkay Altintas 1, Parviez R. Hosseini 2, Derik Barseghian 2, Daniel Crawl
More informationPrototyping an in-situ visualisation mini-app for the LFRic Project
Prototyping an in-situ visualisation mini-app for the LFRic Project Samantha V. Adams, Wolfgang Hayek 18th Workshop on high performance computing in meteorology 24 th -28 th September 2018, ECMWF, UK.
More informationTeam-Based Collaboration in Simulink Chris Fillyaw Application Engineer Detroit, MI
Team-Based Collaboration in Simulink Chris Fillyaw Application Engineer Detroit, MI 2012 The MathWorks, Inc. Development of a complex system Agenda Team-based workflow considerations Reproducing the design
More informationTools and Services for Distributed Knowledge Discovery on Grids
Tools and Services for Distributed Knowledge Discovery on Grids Domenico Talia (Joint work with Mario Cannataro and Paolo Trunfio) DEIS University of Calabria, ITALY talia@deis.unical.it HPC 2002, Cetraro,
More informationCurrent Progress of Grid Project in KMA
Current Progress of Grid Project in KMA CUG 2006 Kim, Hee-Sik Cray Korea Inc. This Presentation May Contain Some Preliminary Information, Subject To Change Outline KMA s Cray X1E system Relationship between
More informationCCSM Performance with the New Coupler, cpl6
CCSM Performance with the New Coupler, cpl6 Tony Craig Brian Kauffman Tom Bettge National Center for Atmospheric Research Jay Larson Rob Jacob Everest Ong Argonne National Laboratory Chris Ding Helen He
More informationPython ecosystem for scientific computing with ABINIT: challenges and opportunities. M. Giantomassi and the AbiPy group
Python ecosystem for scientific computing with ABINIT: challenges and opportunities M. Giantomassi and the AbiPy group Frejus, May 9, 2017 Python package for: generating input files automatically post-processing
More informationLong-term preservation for INSPIRE: a metadata framework and geo-portal implementation
Long-term preservation for INSPIRE: a metadata framework and geo-portal implementation INSPIRE 2010, KRAKOW Dr. Arif Shaon, Dr. Andrew Woolf (e-science, Science and Technology Facilities Council, UK) 3
More informationRDF and Digital Libraries
RDF and Digital Libraries Conventions for Resource Description in the Internet Commons Stuart Weibel purl.org/net/weibel December 1998 Outline of Today s Talk Motivations for developing new conventions
More informationInca as Monitoring. Kavin Kumar Palanisamy Indiana University Bloomington
Inca as Monitoring Kavin Kumar Palanisamy Indiana University Bloomington Abstract Grids are built with multiple complex and interdependent systems to provide better resources. It is necessary that the
More informationAMGA metadata catalogue system
AMGA metadata catalogue system Hurng-Chun Lee ACGrid School, Hanoi, Vietnam www.eu-egee.org EGEE and glite are registered trademarks Outline AMGA overview AMGA Background and Motivation for AMGA Interface,
More informationRunning the model in production mode: using the queue.
Running the model in production mode: using the queue. 1) Codes are executed with run scripts. These are shell script text files that set up the individual runs and execute the code. The scripts will seem
More informationRemote & Collaborative Visualization. Texas Advanced Computing Center
Remote & Collaborative Visualization Texas Advanced Computing Center TACC Remote Visualization Systems Longhorn NSF XD Dell Visualization Cluster 256 nodes, each 8 cores, 48 GB (or 144 GB) memory, 2 NVIDIA
More informationA Dream of Software Engineers -- Service Orientation and Cloud Computing
JICSIT2011 / ITAIC 2011 Keynote http://www.jicsit.org/ A Dream of Software Engineers -- Service Orientation and Cloud Computing Yinong Chen Arizona State University, Tempe, Arizona, U.S.A. JICSIT/ITAIC
More informationUsing the Sakai Collaborative Toolkit in e-research Applications
Using the Sakai Collaborative Toolkit in e-research Applications Charles Severance, Joseph Hardin, Glenn Golden University of Michigan - Sakai Project, Ann Arbor, MI US csev@umich.edu, hardin@umich.edu,
More informationCESM Tutorial. NCAR Climate and Global Dynamics Laboratory. CESM 2.0 CESM1.2.x and previous (see earlier tutorials) Alice Bertini
CESM Tutorial NCAR Climate and Global Dynamics Laboratory CESM 2.0 CESM1.2.x and previous (see earlier tutorials) Alice Bertini NCAR is sponsored by the National Science Foundation Outline The CESM webpage
More informationRENKU - Reproduce, Reuse, Recycle Research. Rok Roškar and the SDSC Renku team
RENKU - Reproduce, Reuse, Recycle Research Rok Roškar and the SDSC Renku team Renku-Reana workshop @ CERN 26.06.2018 Goals of Renku 1. Provide the means to create reproducible data science 2. Facilitate
More informationGeant4 in a Distributed Computing Environment
Geant4 in a Distributed Computing Environment S. Guatelli 1, P. Mendez Lorenzo 2, J. Moscicki 2, M.G. Pia 1 1. INFN Genova, Italy, 2. CERN, Geneva, Switzerland Geant4 2005 10 th user conference and collaboration
More informationSoftware Infrastructure for Data Assimilation: Object Oriented Prediction System
Software Infrastructure for Data Assimilation: Object Oriented Prediction System Yannick Trémolet ECMWF Blueprints for Next-Generation Data Assimilation Systems, Boulder, March 2016 Why OOPS? Y. Trémolet
More informationWeb Interface to Materials Simulations
Web Interface to Materials Simulations Web Interface Generator and Legacy Application Façade Portal Development Team Funding Akos J. Czikmantory (JPL - Wiglaf) DARPA-PROM Hook Hua (JPL - Wiglaf ) JPL SRRF
More informationclimate4impact.eu Christian Pagé, CERFACS
IS-ENES2 1 st General Assembly 11-13 th June 2014 UPC Campus, Barcelona, Spain Status of infrastructure climate4impact.eu Christian Pagé, CERFACS Working teams and institutions CERFACS: Christian Pagé
More informationUsing Resources of Multiple Grids with the Grid Service Provider. Micha?Kosiedowski
Using Resources of Multiple Grids with the Grid Service Provider Micha?Kosiedowski Grid Service Provider The Grid Service Provider came as a result of research done within the PROGRESS project: Project
More informationInstall your scientific software stack easily with Spack
Install your scientific software stack easily with Spack Les mardis du développement technologique Florent Pruvost (SED) Outline 1. Context 2. Features overview 3. In practice 4. Some feedback Florent
More informationWrite a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical
Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or
More informationBy Ian Foster. Zhifeng Yun
By Ian Foster Zhifeng Yun Outline Introduction Globus Architecture Globus Software Details Dev.Globus Community Summary Future Readings Introduction Globus Toolkit v4 is the work of many Globus Alliance
More informationUCT Application Development Lifecycle. UCT Business Applications
UCT Business Applications Page i Table of Contents Planning Phase... 1 Analysis Phase... 2 Design Phase... 3 Implementation Phase... 4 Software Development... 4 Product Testing... 5 Product Implementation...
More informationHIGH PERFORMANCE COMPUTING (PLATFORMS) SECURITY AND OPERATIONS
HIGH PERFORMANCE COMPUTING (PLATFORMS) SECURITY AND OPERATIONS AT PITT Kim F. Wong Center for Research Computing SAC-PA, June 22, 2017 Our service The mission of the Center for Research Computing is to
More informationOperations Orchestration 10.x Flow Authoring (OO220)
Operations Orchestration 10.x Flow Authoring (OO220) Education Services course product number H4S75S Course length 4 days Delivery mode Instructor Led Training (ILT) virtual Instructor Led Training (ILT)
More informationIntroduction on Science Gateway
Introduction on Science Gateway Understanding access and functionalities Riccardo Rotondo riccardo.rotondo@ct.infn.it Catania, 10/03/2014 Outline } What is a Science Gateway? } Science Gateways } General
More informationA High-Level Distributed Execution Framework for Scientific Workflows
A High-Level Distributed Execution Framework for Scientific Workflows Jianwu Wang 1, Ilkay Altintas 1, Chad Berkley 2, Lucas Gilbert 1, Matthew B. Jones 2 1 San Diego Supercomputer Center, UCSD, U.S.A.
More informationCODE AND DATA MANAGEMENT. Toni Rosati Lynn Yarmey
CODE AND DATA MANAGEMENT Toni Rosati Lynn Yarmey Data Management is Important! Because Reproducibility is the foundation of science Journals are starting to require data deposit You want to get credit
More informationA simple OASIS interface for CESM E. Maisonnave TR/CMGC/11/63
A simple OASIS interface for CESM E. Maisonnave TR/CMGC/11/63 Index Strategy... 4 Implementation... 6 Advantages... 6 Current limitations... 7 Annex 1: OASIS3 interface implementation on CESM... 9 Annex
More informationNUSGRID a computational grid at NUS
NUSGRID a computational grid at NUS Grace Foo (SVU/Academic Computing, Computer Centre) SVU is leading an initiative to set up a campus wide computational grid prototype at NUS. The initiative arose out
More informationExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P.
ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. Kushner, D. Waliser, S. Pascoe, A. Stephens, P. Kershaw,
More informationWorkflow Exchange and Archival: The KSW File and the Kepler Object Manager. Shawn Bowers (For Chad Berkley & Matt Jones)
Workflow Exchange and Archival: The KSW File and the Shawn Bowers (For Chad Berkley & Matt Jones) University of California, Davis May, 2005 Outline 1. The 2. Archival and Exchange via KSW Files 3. Object
More informationA More Realistic Way of Stressing the End-to-end I/O System
A More Realistic Way of Stressing the End-to-end I/O System Verónica G. Vergara Larrea Sarp Oral Dustin Leverman Hai Ah Nam Feiyi Wang James Simmons CUG 2015 April 29, 2015 Chicago, IL ORNL is managed
More informationIntroduction to the Europeana SIP CREATOR
Introduction to the Europeana SIP CREATOR Alicia Ackerman alicia.ackerman@kb.nl Development at Europeana Labs with Serkan Demirel serkan@blackbuilt.nl, Initial development by Gerald De Jong Collaboration
More informationPortal Application Deployment Scripting
Portal Application Deployment Scripting Graham Harper, IBM ISSL Senior Application Architect Contents Deployment scripting in context What is a portal application? Portal application components Applying
More informationLinking datasets with user commentary, annotations and publications: the CHARMe project
Linking datasets with user commentary, annotations and publications: the CHARMe project Jon Blower j.d.blower@reading.ac.uk University of Reading On behalf of all CHARMe partners! http://www.charme.org.uk
More informationThe Future of ESGF. in the context of ENES Strategy
The Future of ESGF in the context of ENES Strategy With a subtext of the important role of IS-ENES2 In addressing solutions to the following question: Two thirds of data written is never read! WHY NOT?
More informationHybrid-Type Extensions for Actor-Oriented Modeling (a.k.a. Semantic Data-types for Kepler) Shawn Bowers & Bertram Ludäscher
Hybrid-Type Extensions for Actor-Oriented Modeling (a.k.a. Semantic Data-types for Kepler) Shawn Bowers & Bertram Ludäscher University of alifornia, Davis Genome enter & S Dept. May, 2005 Outline 1. Hybrid
More informationReproducible & Transparent Computational Science with Galaxy. Jeremy Goecks The Galaxy Team
Reproducible & Transparent Computational Science with Galaxy Jeremy Goecks The Galaxy Team 1 Doing Good Science Previous talks: performing an analysis setting up and scaling Galaxy adding tools libraries
More informationOpenSees on Teragrid
OpenSees on Teragrid Frank McKenna UC Berkeley OpenSees Parallel Workshop Berkeley, CA What isteragrid? An NSF sponsored computational science facility supported through a partnership of 13 institutions.
More informationPegasus Workflow Management System. Gideon Juve. USC Informa3on Sciences Ins3tute
Pegasus Workflow Management System Gideon Juve USC Informa3on Sciences Ins3tute Scientific Workflows Orchestrate complex, multi-stage scientific computations Often expressed as directed acyclic graphs
More informationExArch, Edinburgh, March 2014
ExArch: Climate analytics on distributed exascale data archives Martin Juckes, V. Balaji, B.N. Lawrence, M. Lautenschlager, S. Denvil, G. Aloisio, P. Kushner, D. Waliser, S. Pascoe, A. Stephens, P. Kershaw,
More informationHigh Throughput Urgent Computing
Condor Week 2008 High Throughput Urgent Computing Jason Cope jason.cope@colorado.edu Project Collaborators Argonne National Laboratory / University of Chicago Pete Beckman Suman Nadella Nick Trebon University
More informationEnhancement for bitwise identical reproducibility of Earth system modeling on the C-Coupler platform
Geosci. Model Dev. Discuss., 8, 2403 243, www.geosci-model-dev-discuss.net/8/2403// doi:.194/gmdd-8-2403- Author(s). CC Attribution 3.0 License. This discussion paper is/has been under review for the journal
More informationManaging Exploratory Workflows
Managing Exploratory Workflows Juliana Freire Claudio T. Silva http://www.sci.utah.edu/~vgc/vistrails/ University of Utah Joint work with: Erik Andersen, Steven P. Callahan, David Koop, Emanuele Santos,
More informationCISL Update. 29 April Operations and Services Division
CISL Update Operations and Services CISL HPC Advisory Panel Meeting Anke Kamrath anke@ucar.edu Operations and Services Division Computational and Information Systems Laboratory 1 CHAP Meeting 14 May 2009
More informationUsing the Eclipse Parallel Tools Platform in Support of Earth Sciences High Performance Computing
Using the Eclipse Parallel Tools Platform in Support of Earth Sciences High Performance Computing Jay Alameda National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign
More informationExperiences with Porting CESM to ARCHER
Experiences with Porting CESM to ARCHER ARCHER Technical Forum Webinar, 25th February, 2015 Gavin J. Pringle 25 February 2015 ARCHER Technical Forum Webinar Overview of talk Overview of the associated
More informationSouth African Science Gateways
Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing Research Infrastructures Grant Agreement n. 306819 South African Science Gateways Bruce Becker, Coordinator,
More informationThe challenge of porting scientific results to operational applications
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss The challenge of porting scientific results to operational applications Rebekka Posselt*, Rebecca Hiller,
More informationTHE IMPORTANCE OF NICHE TECHNOLOGIES IN BUSINESS ANALYSIS. - Kat Okwera Jan 2019
THE IMPORTANCE OF NICHE TECHNOLOGIES IN BUSINESS ANALYSIS - Kat Okwera Jan 2019 HEY THERE I M A BA TOO! Kat Okwera Programmer Systems Designer Web Developer Project Manager Business Analyst E-Learning
More information