Data discovery and access via the SeaDataNet CDI system

Transcription

1 Data discovery and access via the SeaDataNet CDI system Central dataproducts and data services on distributed data. Peter Thijsse MARIS CLIPC IS-ENES workshop, KNMI, November 2014

2 Outline 1. Introduction to Common Data Index (CDI) system 2. Options of metadata and data access: a) WMS / WFS b) GI-CAT CSW c) OpenSearch d) Using a dedicated buffer and services 3. Discussion

3 1. Introduction to CDI

4 Common Data Index (CDI) Data Discovery and Access Service is a fine-grained index (ISO / ISO 19139) to individual data measurements (such as a CTD cast or moored instrument record). The CDI Data Discovery and Access Service provides users a highly detailed insight and unified access to the large volumes of marine and oceanographic data sets managed by the distributed data centres An intelligent middle tier connection is configured between the SeaDataNet portal and the local data management systems at each of the data centres. A shopping basket allows users to submit a shopping request for multiple data providers in one go and to follow its processing by each of the providers via an online transaction register.

5 Delivery Services CDI V3 architecture

6 All directories support the steps from data discovery to unified data access and delivery

7 Common Data Index (CDI) data discovery and access service of archived data quality checked data.

8 > 1,6 million CDI entries from 34 countries and >100 data centres and 487 originators for physics, chemistry, geology, geophysics, bathymetry and biology; years ; 85% unrestricted or under SeaDataNet licence

9 2. Interoperability options

10 OGC WMS / WFS CDI supports OGC WMS v1.3 and WFS v1.1 services Horizontal exchange of geographical coverage of metadata Applicable in any map viewer WFS points back for more metadata and request for data acces to the central CDI system /seadatanet/cdi_v2/seadatanet

11 GICAT / CSW Horizontal exchange to overarching portals CDI database metadata divided in parts: per discipline per datacenter Each part described with ISO19139 metadata and Supplied via CSW or OAI-PMH protocol A.o. available in Ocean Data Portal and GEOSS

12 OpenSearch Now available for horizontal exchange of metadata Simple search options: Free text Geographical search (radius, box) Temporal search (date from/to) Response in JSON/Atom RSS. Available on virtual subsets (entry points similar to CSW) of CDI database Link per subset leads exactly to search result in CDI interface rch/os_description

13 Use of dedicated data buffers External portals could make use of a data buffer. According to a specific profile (Specify: datatype, region, datacenters, etc) in the SeaDataNet buffer system a subset is created. The SDN robot builds up a (consistently updated) specific subset of data relevant for external portal. To create this buffer an agreement of involved data centers is required. => Download only for creating dataproducts and after agreement of data owner. Not allowed to redistribute the data, other than via CDI system and download directly from partner

14 Example use of data buffer

15 Services on top of buffer Dedicated user interface for quick retrieval of data (no need to wait for order system and/or confirmation of datacenter). User are registered SDN users and activity is measured. WMS/WFS possible on top of subset for integration in external portal. OpenSearch option for integration in external portal. WMS/WFS and OpenSearch point back to CDI portal for data order. Machine to machine services API to be used to create dataproduct: Create_order: enabling to configure a discovery profile for shopping in an authorised central buffer database; Get_order_info: enabling to retrieve info from the RSM about the progress of the order processing; Download_order: enabling to download the orders, when ready, as zip files with metadata and data sets. Services secured by SDN CAS (AAA) login

16 EMODNet and SeaDataNet action results: Visualisation of climatologies on top of validated buffer

17 Regional products (Internal SDN product group): Temperature Baltic Sea

18 Regional products (SDN product group): Temperature North Atlantic Ocean

19 Regional products EMODNet: Example catalogue EMODNet Chemistry

20 Regional products EMODNet: Visualisation Seasonal nitrate ( ) from catalogue EMODNet Chemistry

21 Specifications Previous products are developed using a.o.: Thematic and regional buffers (data collections) Harmonised raw datasets in ODV format (some NetCDF) using ODV software for aggregation: DIVA reanalysis software (Univ of Liege): Oceanbrowser visualisation software (Univ. of Liege), using OGC standards, NetCDF files Sextant data product catalogue (IFREMER) based on GeoNetwork (able to be accessed via CSW, OAI-PMH), products described using SeaDataNet vocabularies where possible.

22 THANK YOU!