Data and visualization Harry Lankreijer Joint ICOS Carbon Portal and EUROCOM Workshop 2016-06-21 ICOS Carbon Portal Lund University
Jena inversion, Christian Rödenbeck; at CP webpage (https://www.icos-cp.eu/node/52) ICOS CP services/visualizations: - Spatial data THREDDS server - Time series data - Flux estimates - Footprint tool - STILT-simulated, CO2 time series
Metadata? To know what the data is about - Information on the characteristics of the data: eg quality (limitations and possibilities): easily understood by people who might want to use your data - To keep track of the history of the data: how was it processed, gap filled, instruments (observation data), model version, input data etc. (provenance) - To make data discoverable and searchable: the users need to be able to find datasets that meet their qualifications. To locate it from a citation - To make data compatible and exchangeable - To make data citable and keep track of the use of the datasets - Transparency
Metadata A metadata profile (or extension) needs to be defined and developed through experience A profile is a set of metadata elements (fields, terms, attributes) following a common language; needs a standard
ICOS will follow the EU directive of INSPIRE and thus the INSPIRE metadata standard INSPIRE: Infrastructure for Spatial Information in the European Community INSPIRE: http://inspire.ec.europa.eu/ INSPIRE Geoportal: http://inspire-geoportal.ec.europa.eu/ INSPIRE prescribes implementation rules as defined by the ISO standards for geographical information: ISO/TC 211-19115 Geographic information - Metadata
Mandatory, conditional, optional and data specific elements Metadata profile or extension Conditional Mandatory Optional Metadata standard
Another standard of interest: The Dublin Core Metadata Initiative = ISO 15836 standard (DC) Original 15 basic elements; More refined version: 55 elements Title, Creator, Subject Description Publisher, Contributor Date, Type, Format Identifier Source, Language, Relation Coverage Rights SpatialResolution, Temporal, Overview: http://purl.org/dc/terms/
Test case: filling out the required elements for Dublin Core what exactly is meant? how many different items per element Element File name Contributor Creator Value s99_v3.7_monthly Roedenbeck, Christian, MPI Biogeochemistry, Jena, Germany, Christian.Roedenbeck@bgc-jena.mpg.de, ORCID: N/A Roedenbeck, Christian, MPI Biogeochemistry, Jena, Germany, Christian.Roedenbeck@bgc-jena.mpg.de, ORCID: N/A Coverage Global, 1999-2014 Spatial -180 -- +180; -90 -- +90; lon-lat SpatialResolut ion 5, 3.75 MACC (F. Chevallier) Jena Inversion System (C. Rödenbeck) Carbon Tracker Europe (I. van der Laan-Luijkx) Temporal 1999-01-15, 2014-12-15 TemporalReso lution monthly Date N/A Description Jena inversion results; Jena Carboscope; http://www.bgcjena.mpg.de/~christian.roedenbeck/download-co2/
More rules on keywords and standards: INSPIRE metadata elements: CP tool for definition of the metadata 1. Identification 2. Classification of spatial data and services 3. Keyword(s) 4. Geographic location 5. Temporal reference (time span, date of publication) 6. Quality and validity 7. Conformity (technical arrangements for the interoperability) 8. Constraint related to access and use (license) 9. Organizations responsible for the data 10.Metadata on metadata
Metadata editor: http://inspire-geoportal.ec.europa.eu/editor/ Generates a xml-file of the metadata
Mandatory profile items, INSPIRE based on ISO19115 INSPIRE/ISO19115 1.1 Resource title: name of resource 1.2 Resource abstract: description of resource 1.3 Resource type: dataset, series or service 1.4 Resource locator: URL 1.5 Unique resource identifier: DOI or other PID 1.7 Resource language: Eng 2.1 Topic category: select from list 3 Keywords: both INSPIRE defined, GEOSS and ICOS 4.1 Geographic bounding box 5 Temporal reference 6.1 Lineage (process history, validation, quality) Here is not the question which metadata elements But what is meant and what is needed (Test for different datasets will help to develop the tool) 6.2 Spatial resolution (Conditional); scale 1:x, 7 Conformity (for the CP to handle) 8.1 Conditions for access and use 8.2 Limitations on public access 9 Responsible organisation: different org/ roles can be given; ORCHID 10.1 Metadata point of contact: CP 10.2 Metadata date: date publication 10.3 Metadata language: eng
The question now is: what is needed to describe/present the products (requirements from ICOS) and which other metadata elements do we need: If we have an inverse modelling result/series as a product to be visualized: - model version; software - input data - parameters settings - boundary conditions for regional models - information to increase findability
Summary slide on DOI/Maggie 1. Important to get a persistent identifier for data sets that should be referable especially finalized data sets 2. DataCite DOI most useful? (High visibility, comprehensive metadata,...) 3. Citing data: need to engage the science community to identify practices & needs 4. Bibliometry: what usage reports does the community need? 5. Dynamic data : is this relevant to the modelling community? 6. Data lifecycle: use PIDs to inform workflow engines; provenance tracing June 20-22, 2016 Joint ICOS Carbon Portal & EUROCOM workshop 13
Development of the Workflow at Carbon Portal: a test case Modeller data model results (eg. netcdf CF 1.4) metadata additional information: model set-up input data Carbon Portal extract from netcdf file extract from database data metadata metadata database Carbon Portal website store search visualize document CP-upload tool Example: Inversions in Global Carbon Project MACC (F. Chevallier) Jena Inversion System (C. Rödenbeck) Carbon Tracker Europe (I. van der Laan-Luijkx) Project/landingpage
The ISO191xx series: Series or set of geographic information standards associated with a location relative to the Earth (ISO 19101). developed by the ISO Technical Committee 211 (http://www.isotc211.org/ http://www.iso.org/ ). The project 19115 metadata was initiated in 1996 and published as ISO standard in May 2003 Several other relevant standards in the ISO 191xx series: ISO 19119 Geographic information - Services ISO 19106 How to make a metadata profile ISO 19139 xml encoding of ISO 19115 ISO 19156:2011 and Open Geospatial Consortium (OGC) on Geographic information Observations and measurements