European and international background

Transcription

1 IS-ENES2 General Assembly 11-13th June 2014 Barcelona European and international background Metadata and ES-DOC (including statistical downscaling) Sébastien Denvil, Mark Greenslade, Allyn Treshansky, Sylvia Murphy, Stephen Pascoe, Martina Stockause Eric Guilyardi, Bryan Lawrence, Cecelia de Luca et al.

2 What is it about? Metadata context, CIM and es-doc Downscaling evolution Future plan and requirements Simplification From Long Term Archiving From modelling center

3 To keep in mind the potential to interpret, compare and reuse climate information results is strongly related to the quality of their description But metadata alone won't get us there! Computation useless if results cannot be stored/distributed/read

4 Hype.vs. Reality Interoperability in Earth Sciences Semantic Web OpenSearch Web Map Service OPeNDAP CIM Web Coverage Services NetCDF CF-1 GCMD DIF HDF-EOS ECHO Catalog Services for the Web Technology Trigger Peak of Inflated Expectations Trough of Disillusionment Slope of Enlightenment Plateau of Productivity

5 EU - IPSL, BADC, DKRZ US - NOAA, NCAR, PCMDI

6

7 A climate simulation Why What How Experiment Simulation Model Input: Coupling Output: 1..* Requirement 0..* Conformance Software Component Name Properties Description Coupling Framework 0..1 Parent 0..* Child

8 CIM The CIM is intentionally very general. It can be customized for particular user communities through the addition of specific Controlled Vocabularies. A Controlled Vocabulary defines the content that can be used within a CIM document. For example, in the case of climate models, the CIM schema (structure) allows a ModelComponent to have a child ModelComponent. And each of those components can have types. A CV is required to list the permitted types. For example, the CMIP5 CV allows an atmosphere model to have a child advection model, but not a child ocean model. Thus, in order to be valid a CIM document must conform both to the CIM schema and to a particular set of CVs.

9 Downscaling Metadata recent evolutions NCPP has been extending the CIM for downscaling NCPP presented an initial proposal to modify the CIM in order to accommodate a description of downscaling methods. It deals mostly with the conceptual level how downscaling concepts map onto the structures available in the CIM. They made several specific suggestions for modifications to the CIM, and identify some conceptual issues that require further thought and consideration by the larger community.

10 Downscaling Metadata recent evolutions NCPP has been extending the CIM for downscaling adding additional structures to CIM Schema: re-structuring and adding content to the CV for statistical downscaling: branches/cmip5/software/atmosphere_bdl.mm re-using CMIP5 CV for dynamical downscaling: branches/ncpp/downscaling/statistical_downscaling_ncpp.mm

11 Future plans for the ecosystem --> simplify

12 Informs ConCIM (UML) MindMap Vocabulary Generates Generates Informs ConCIM ESDOC-mp Python Ontology AppCIM Generates AppCIM (XSD) CV Canonical XML Utilises Utilises CMIP5q/PIMMS (Django) Creates ESDOCq (Django) Creates ESDOC XML/JSON (doc) Metafor XML (doc) Harvests Viewer Creates ESDOC Repositories Harvests Comparator pyesdoc (python library) HTML (doc) Creation Content Storage Tools

13 Informs ConCIM (UML) MindMap Vocabulary Generates Generates Should Generate CV Canonical XML Creates ESDOCq (Django) Creates Should Produce Creates ESDOC XML/JSON (doc) Metafor XML (doc) Harvests Viewer ESDOC-mp Python Ontology AppCIM Should Utilise Utilises CMIP5q/PIMMS (Django) ConCIM Ontology migration & tools?? Generates AppCIM (XSD) Informs ESDOC Repositories Harvests Comparator Utilises pyesdoc (python library) Creation Should be able to produce HTML (doc) Needs to standalone validate Validation Tool PDF (doc) Content Storage Tools

14 MindMap Vocabulary ConCIM (UML) Informs ConCIM Generates CV Canonical Format TBD ESDOC-mp Python Ontology AppCIM Utilises CMIP5q/PIMMS (Django) ESDOCq (Django) pyesdoc (python library) Creation Creates Harvests ESDOC XML/JSON (doc) Validates ESDOC Repositories Viewer PDF (doc) HTML (doc) Comparator Validation Tool Content Storage Tools

15 Requirements from modelling groups

16 CMIP5 Model/Experiment Metadata ES-DOC/ESGF CMIP5 Questionnaire CREM CREM screen scrape hmm hmm cut Science Teams across UK Crown copyright Met Office paste

17 CMIP6 Model/Experiment Metadata - planned CREM/ROSE CREM/ROSE CIM Metadata Formatter ESGF/ES-DOC ESGF/ES-DOC Portal Portal Validate Publish Redact/Update review documents Science Teams across UK Crown copyright Met Office MOHC IS-ENES-2 Development

18 Requirements from LTA side (Long Term Archiving)

19 LTA Workflow with CIM and QC Quality CIM Quality Quality Docs Results CIM Docs Harvesting / CIM Viewer Display 4b DOI/Citation Information 2c Questionnaire CIM Docs 3b Index Node 2a Metadata Node Portal Ingest Node Node Scientist Citation Information ESDOC Node Metadata Enrichment 2b 1 Replication Archival Metadata 4a DOI process 3a DOI / Citation Information + other catalogs ESGF: Project Repository DKRZ WDCC: Long-term Archive 1 Replicate 2 LTA 3 Cite DOI 4 Share DOI 19

20 Ideal LTA Workflow 4b DOI/Citation Information of IPCC-DDC data ESCIM Docs DOC CIM Doc ID ingest Node Index Node Node Node Node 2a Metadata Ingest (incl. IDs to add CIM information) 2b 1 Replication Archival Portal Metadata 4a + other catalogs DOI / DOI process Citation Information 3a ESGF: Project Repository DKRZ WDCC: Long-term Archive 1 Replicate 2 LTA 3 Cite DOI 4 Share DOI

21 Thank you for your attention

22 Existing ES-DOC services A guided tour

23 ES-DOC Tools - Document Viewer

24

25

26 ES-DOC API - Web Service Create PUBLISH Retrieve Update Delete ID / version Type / name SEARCH set id File id DRS COMPARE C1 = Model component properties