IMPLEMENTING THE WASCAL DATA INFRASTRUCTURE (WADI)

Size: px

Start display at page:

Download "IMPLEMENTING THE WASCAL DATA INFRASTRUCTURE (WADI)"

Morgan Horton
5 years ago
Views:

1 IMPLEMENTING THE WASCAL DATA INFRASTRUCTURE (WADI) Ralf Kunkel, Antonio Rogmann, Jürgen Sorg, Huaping Wang Helmholtz Open Science Webinare zu Forschungsdaten,

2 What is WASCAL? West African Science Service Center on Climate Change and Adapted Land Use aims at strengthening the research, educaonal and policy capacity and competence of West- African countries deals with issues of climate change through adapted land use on a scienfic basis partnership with German instuons (research instutes, universies) Implementaon phase ( ) funded by the German Federal Ministry of Educaon and Research WASCAL was recognized by ECOWAS as an internaonal organizaon in May countries signed WASCAL s constuon so far! Benin! Burkina Faso! Côte d Ivoire! The Gambia! Ghana! Mali! Niger! Nigeria! Sénégal! Togo!

WASCAL Pillars Graduate Program Supports and facilitates academic educaon (PhD and Master students) amongst West African universies in associaon with German counterpart instuons.

3 WASCAL Pillars Graduate Program Supports and facilitates academic educaon (PhD and Master students) amongst West African universies in associaon with German counterpart instuons. Core Research Program German/African joint research program on adapted land use and management of land under changing climac condions. Competence Center (Ouagadougou) Accommodates the infrastructure for data recepon, data maintenance and access, data interpretaon and scenario analyses Offers experse necessary for analysing the impacts of climate change and for developing strategies and policies Provides a collaborave work environment for the scienfic staff, administrave and support personnel, and vising sciensts Conducts training and outreach acvies to involve the stakeholders Connects regional partners in data collecon networks (observaon networks)

4 WASCAL data infrastructure (WADI) requirements Common data sharing policy Interoperability and standards Graduate program research results Data provision Data publicaon Files Maps Time series Model results Core research program results Observaon network and remote sensing data Climate model results Hydrological model results Socio- economical model results

WADI components WASCAL geoportal Metadata Catalog WADI Metadata

research results BUTT Service Static geodata (Geoserver) PostgreSQL

OGCcompliant services Observation data (TSM) OGC Web Services

socio-economical, and biodiversity observation networks Remote

sensing data (DIMS) External data users Product management Quality

5 WADI components WASCAL geoportal Metadata Catalog WADI Metadata (Geonetwork) PostgreSQL File based data (BUTT) Graduate program research results BUTT Service Static geodata (Geoserver) PostgreSQL Files" Maps" Time"series" ^" Core research program results OGCcompliant services Observation data (TSM) OGC Web Services Model"results" Data from the hydrological, meteorological, socio-economical, and biodiversity observation networks Remote sensing data Data Data discovery Discovery Portal portal Remote sensing data (DIMS) External data users Product management Quality control database Data import Climate model results Hydrological model results Socio-economical model results

6 Data and Metadata Metadata are data providing informaon about one or more aspects of the data, such as: Means of creaon of the data Purpose of the data Time and date of creaon Creator or author of the data Locaon where the data were created Standards to create or describe the data Required to describe, discover and specifying the locaon of available data No data without metadata Each data set is described by a metadata set No metadata without data (if possible) Each metadata set contains the locaon of the data

Data and Metadata Metadata are data providing informaon about one or more aspects of the data, such as: Means of creaon of the data Purpose of the data Time and date of creaon Creator or author of

7 Data and Metadata Metadata are data providing informaon about one or more aspects of the data, such as: Means of creaon of the data Purpose of the data Time and date of creaon Creator or author of the data Locaon where the data were created Standards to create or describe the data Required to describe, discover and specifying the locaon of available data No data without metadata Each data set is described by a metadata set No metadata without data (if possible) Each metadata set contains the locaon of the data

8 Data and Metadata Metadata are data providing informaon about one or more aspects of the data, such as: Means of creaon of the data Purpose of the data Time and date of creaon Creator or author of the data Locaon where the data were created Standards to create or describe the data Required to describe, discover and specifying the locaon of available data No data without metadata Each data set is described by a metadata set No metadata without data (if possible) Each metadata set contains the locaon of the data } allowed No divorce

9 Pung metadata and data files into WADI Online Metadata eding File data upload and download Metadata query and discovery Advanced visualizaon of geospaal layers WASCAL Geoportal (geonetwork) Metadata and data data Metadata storage Data storage Bulk upload and data transfer tool (BUTT) Offline metadata eding Desktop applicaon for large size file data upload

10 Accessing WASCAL data hp:// Gateway to data for stakeholders, politicians and scientists Search and discover data catalogues Predefined queries (e.g. soil ) Custom multi-conditional queries

11 Accessing WASCAL data hp:// Gateway to data for stakeholders, politicians and scientists Search and discover data catalogues Predefined queries (e.g. soil ) Custom multi-conditional queries Provides detailed information on data sets Advanced visualization of geospatial layers through WASCAL geoportal Online resources provides direct access to data 11

Observaon networks and databases Within WASCAL observation networks and databases will be established in cooperation with national services: Meteorology

12 Observaon networks and databases Within WASCAL observation networks and databases will be established in cooperation with national services: Meteorology (station network) Hydrology (station network) Land (satellite receiving station) Biodiversity (observatories, surveys) Households (socio-economic surveys)

13 Observaon network (me series) data Hydro- /Met services Country A Data sources: Sent automatically (e.g. by GSM, ) Retrieved from station manually by technician Historical or preprocessed data already present Typical data file Data exchange Competence Center Hydro- /Met services Country B Data will be stored in a data base rather than in individual files

14 Steps towards a distributed data infrastructure Status of local data infrastructures Level 1: No central file server, mostly hardcopy data or file based data (e.g. Excel sheets) on local machines Level 2: Central file server and/or data base, no publicaon of data to the web Level 3: Central file server and/or data base, applicaon server to publish data to the web Support by WASCAL Digizing hardcopy data Uploading data into a central WADI database Unfied data access by WADI (set up local data bases) Seng up web services to publish data via WADI (or any other external web clients) Seng up own web interfaces to provide data to the web

15 Challenges to implement observaon neworks in West- Africa Current situaon: Lack of technical and human resources Exisng staons oen need to be improved to allow automated data transfer Historical data exist only in analogous form or file based (Excel- sheets) No common internal data infrastructure (central server), almost no metadata available Somemes necessity for governmental instuons to sell data Level 1 Situaon Measures (WASCAL ReHCON workshop , Ouaga) Acquision of unified equipment by WASCAL and transfer to each Naonal Service (NHMS) Training of staff such as observers, meteorological technicians and engineers Support to digize historical data Implemenng the transfer of present and past hydrological observaons from the NHMS to the Competence Centre and vice Parcipave implementaon of regional plaorms connecng NHMS and regional researches at the Competence Centre

Imporng and processing observatory network data into WADI Automated data import and pre- processing Processing of different import sources (E- Mail, ASCII- files ) Min/Max- filter for data

16 Imporng and processing observatory network data into WADI Automated data import and pre- processing Processing of different import sources (E- Mail, ASCII- files ) Min/Max- filter for data prevalidaon Applicaon of sensor specific transformaon rules, mean values, temporal aggregates Data archiving OGC- conformal web services for the coupling to the data portal WADI%Portal% WADI Observatory Data Portal

17 Accessing WASCAL me series data Standardized, multiconditional search Visualization and download 17

18 Interoperability and standards: Data management plan Outlines how data are handled during research and after the project is completed. Described issues: Technical implementation of the data infrastructure Data to be managed Used standards (services, data, quality, metadata, ) Data preservation and security strategies Organizational framework Ensures that data are wellmanaged in the present, and prepared for long term preservation in the future

19 Data sharing policy General directives and rules on how to share, reuse and cite WASCAL data: Ethical Issues and data privacy Written consents to use personal data for WASCAL research Anonymisation of personal data Copyrights and intellectual property (IP) protection Rules for IP and Non-IP-data Rules depend on work contract conditions of IP-data authors Retention times for certain groups Licensing model (Creative Commons) Citation rules Provision, distribution, and reuse of externally owned data set by additional bilateral agreements

authorized users / groups possible: Data eding and flagging Data download (depending on access constraints) With each download:

20 Implemenng data access rights Data security and data access rights defined by The WASCAL data policy document Addional documents and regulaons Central user/group management by LDAP directory (WASCAL home page, Data Discovery Portal, Geoportal) Only for authorized users / groups possible: Data eding and flagging Data download (depending on access constraints) With each download: Data use license provided E- Mail noficaon to responsible party containing informaon with requesng user, downloaded data and purpose

21 Questions? Contact: Ralf Kunkel Antonio Rogmann

DataONE Cyberinfrastructure. Ma# Jones Dave Vieglais Bruce Wilson

DataONE Cyberinfrastructure. Ma# Jones Dave Vieglais Bruce Wilson DataONE Cyberinfrastructure Ma# Jones Dave Vieglais Bruce Wilson Foremost a Federa9on Member Nodes (MNs) Heart of the federa9on Harness the power of local cura9on Coordina9ng Nodes (CNs) Services to link