D4.1 Transcontinental Data Infrastructures and Data repositories

Transcription

1 Project acronym Project full title : CHAIN-REDS Grant agreement : Start date : December 1, 2012 Duration Programme Theme Thematic area Funding scheme Call identifier Project coordinator : Co-ordination & Harmonisation of Advanced and e-infrastructures for Research Education Data Sharing : 30 months : 7th Framework Programme (FP7) : Capacities specific program : Research Infrastructures : Support action : FP7 INFRASTRUCTURES : Federico Ruggieri (INFN) D4.1 Transcontinental Data Infrastructures and Data repositories Deliverable Status : Final File Name : CHAIN-REDS-D4.1_v04.docx Due Date : May 2013 (M06) Submission Date : May 2013 (M06) Dissemination Level : Public Author : CIEMAT (rafael.mayo@ciemat.es) Copyright The CHAIN-REDS Consortium INFN CIEMAT GRNET CESNET UBUNTUNET CLARA IHEP ASREN SIGMA Istituto Nazionale di Fisica Nucleare - Italy Centro de Investigaciones Energeticas, Medioambientales y Tecnologicas - Spain Greek Research and Technology Network S.A. - Greece Zajmove Sdruzeni Pravnickych Osob - Czech Republic The Ubuntunet Alliance for Research and Education Networking - Malawi Cooperacion Latinoamericana de Redes Avanzadas - Uruguay Institute of High Energy Physics, Chinese Academy of Sciences - China Arab States Research and Education Network - Jordan Sigma Orionis - France proj-office@chain-project.eu Grant Agreement n

2 CHAIN-REDS Project - Deliverable D4.1 Page #2 Disclaimer More details on the copyright holders can be found at CHAIN-REDS ( Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing ) is a project co-funded by the European Union in the framework of the 7 th FP for Research and Technological Development, as part of the Capacities specific program - Research Infrastructures FP7 INFRASTRUCTURES For more information on the project, its partners and contributors visit hwww.chain-project.eu. You are permitted to copy and distribute verbatim copies of this document containing this copyright notice, but modifying this document is not allowed. You are permitted to copy this document in whole or in part into other documents if you attach the following reference to the copied elements: "Copyright (C) CHAIN-REDS Consortium - The information contained in this document represents the views of the CHAIN-REDS Consortium as of the date they are published. The CHAIN-REDS Consortium does not guarantee that any information contained herein is error-free, or up to date. THE CHAIN CONSORTIUM MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, BY PUBLISHING THIS DOCUMENT. Revision Control Issue Date Comment Author V01 29/04/13 Frist draft Rafael Mayo-García V02 30/04/13 Some comments Sijin Qian, Rafael Mayo-García V03 13/05/13 New concepts added Ognjen Prjnat, Rafael Mayo-García V04 31/05/13 Final review and corrections Federico Ruggieri 2

3 CHAIN-REDS Project - Deliverable D4.1 Page #3 Abstract CHAIN-REDS project is focused on promoting and supporting technological and scientific collaboration across different e-infrastructures established and operated in various continents. Addressing basic issues such as data persistency, accessibility and interoperability is the related goal for WP4 Data infrastructures. To do so, the adoption of standards is a key point. With regards to it, four (4) of them have been selected in the field of data preservation, access and use and the CHAIN-REDS Knowledge Base has been improved accordingly. Now, it includes DCIs information, as in CHAIN project lifetime, but also (Open Access) Data Repositories. These repositories can be semantically searched by means of web enrichment and the implementation of a semantic engine. Initiatives of interest and some of the repositories appearing in the KB have been obtained from an international survey carried out by WP4 during the months of February and March 2013, the results of which are reported in this deliverable. In addition, CHAIN-REDS has identified several fields in which further collaboration on data aspects will be mutually beneficial: document repositories, agriculture, Earth sciences, e-government and astrophysics. Leading role projects in these fields have been contacted for a closer collaboration. 3

4 CHAIN-REDS Project - Deliverable D4.1 Page #4 Table of contents ABSTRACT 3 TABLE OF CONTENTS 4 PURPOSE 5 GLOSSARY 5 1. INTRODUCTION 7 2. INTERNAL WORK PLAN 8 3. STANDARDS ADOPTED INFORMATION GATHERING AND ACTIONS PERFORMED INFORMATION GATHERING THE CHAIN-REDS KNOWLEDGE BASE IDENTIFIED COMMUNITIES AGRICULTURE AGINFRA E-GOVERNMENT ENGAGE EARTH SCIENCES EARTHSERVER ASTROPHYSICS IVOA GENERAL DATA RELATED INITIATIVES EUDAT AND DOCUMENTS REPOSITORIES 24 CONCLUSIONS 24 4

5 CHAIN-REDS Project - Deliverable D4.1 Page #5 Purpose CHAIN-REDS is a FP7 project co-funded by the European Commission (DG CONNECT) which started on the December 1 st, 2012 and aims at promoting and supporting technological and scientific collaboration across different e- Infrastructures established and operated in various continents, in order to define a path towards a global e-infrastructure ecosystem that will allow Virtual Research Communities (VRCs), research groups and even single researchers to access and efficiently use worldwide distributed resources, i.e. computing, storage, data, services, tools, applications. This deliverable presents the results of the investigation made on the existing and emerging Data Infrastructures and Data repositories that have or could have trans-continental activities and/or associated VRCs in the regions targeted by CHAIN-REDS. This is a first version at month 6 which will have three updates at month 12 (this will be included in D4.2), 18 (D4.3) and 24 (D4.4). This deliverable also documents the results of the survey made by WP4 on Data infrastructures and the standards that have been selected for working on data interoperability. Several scientific fields have been identified for presenting them as a proof-of-principle trust building at the end of the project, which are also described. Glossary CA CHAIN CHAIN-REDS DCI DCMI DoW DR EC EGI EGI-InSPIRE EPIKH FP7 GA HPC IdF IVOA KB Certification Authority Co-ordination and Harmonisation of Advanced e-infrastructures Co-ordination and Harmonisation of Advanced e-infrastructures for Research Education Data Sharing Distributed Computing Infrastructure Dublin Core Metadata Initiative Description of Work Annex I to the GA Data Repository European Commission European Grid Initiative European Grid Initiative-Integrated Sustained Pan-european InfRastructurE Exchange Programme to advanced e-infrastructure Know-How European Commission s Framework Programme Seven Grant Agreement High Performance Computing Identity Federation International Virtual Observatory Alliance Knowledge Base 5

6 CHAIN-REDS Project - Deliverable D4.1 Page #6 KLIOS MoU NGI NREN OADR OAI-PMH OWL RDF ROC VRC WP XML Knowledge Linking and sharing in research domains Memorandum of Understanding National Grid Initiative National Research and Education Network Open Access Data Repository Open Archives Initiative Protocol for Metadata Harvesting Ontology Web Language Resource Description Framework Regional Operation Centre Virtual Research Community Work Package Extensible Markup Language 6

7 CHAIN-REDS Project - Deliverable D4.1 Page #7 1. Introduction CHAIN-REDS isfocused on promoting and supporting technological and scientific collaboration across different e-infrastructures established and operated in various continents. Thus, it aims to facilitate these e-infrastructures uptake and their final use by established and emerging Virtual Research Communities (VRCs), but also by single researchers. To do so, it is essential to promote instruments and practices that can facilitate their inclusion in the community of users, i.e. the use of standards. Then, to build on the best practices currently adopted in Europe and other continents, and to promote and facilitate interoperability among different e-infrastructures is a must. In this way, it will be an asset to work out a step-by-step strategy for developing in the regions targeted by CHAIN-REDS the European data infrastructure gradually. Addressing basic issues such as data persistency, accessibility and interoperability will be the first general goal. CHAIN-REDS, in accordance with several European strategies, plans to focus on including low-level services, exchanging in data infrastructures and support preservation and data exploitation services, as well as on activities aimed at promoting interoperability and data access federation and openness. Building reliable and robust data services suitable to real needs will be also pursued. To do so, stakeholders of the data infrastructures should be involved as well as resource providers and infrastructures, initiatives and user community s coordinators. In this sense, it is worth mentioning the European Union Open Data Portal 1 promoted by the European Commission 2 or the landmark steps to liberate open data already declared by USA President Obama 3. The first step is to identify such best practices and approach the main stakeholders of regional e-infrastructures and of data provisioning/use in order to propose and even define a path towards a global e-infrastructure ecosystem that will allow VRCs, research groups and even single researchers to access and efficiently use worldwide distributed resources, i.e. computing, storage, data, services, tools, applications. As a main issue, the efficient access, use and further analysis of Data has emerged. The number of Data Repositories (DRs), either Open Access ones (OADRs) or not, and the quantity of Terabytes they store have largely increased in the latest years. As a consequence, if CHAIN-REDS aims to allow VRCs, research groups and single researchers to efficiently use worldwide distributed resources, it is needed that the data they are employing will be interoperable as well. Otherwise, advances made on middleware interoperability will result meaningless since the computational resources will not be properly exploited. CHAIN project, the precursor of CHAIN-REDS, already promoted interoperability as a main objective. A worldwide demo was showed and demonstrated in

8 CHAIN-REDS Project - Deliverable D4.1 Page #8 September 2012 and the project carried out several actions that were a step beyond in facilitating the access to information coming from different regions. Thus, the Knowledge Base 4 (KB) provided information about the deployment of e- Infrastructure related topics per country and even about specific Distributed Computing Infrastructures (DCIs) by means of a Site or a Table view. All these concepts conducted to a validation model for VRCs that was successfully tested by the end of CHAIN by counting on the aforementioned worldwide demo and the road-map of services requested from the VRCs to the DCIs. During the first six months of CHAIN-REDS, WP4 Data Infrastructures has been working on extending the CHAIN Knowledge Base with information related to data infrastructures. To do so, it has collected both issues and best practices and has surveyed the involved regions in order to discover data repositories that could be of interest for VRCs. The reason for that is to promote data sharing across different e-infrastructures and continents widening the scope of the existing CHAIN Knowledge Base to Data Infrastructures and to finally provide proof-of principle use-cases for Data sharing across the continents by the end of CHAIN- REDS. This deliverable documents the actions that have been performed to the moment, this is, the results of the investigation made on the existing and emerging Data Infrastructures and Data repositories that have or could have trans-continental activities and/or associated VRCs. 2. Internal work plan Once the CHAIN-REDS Description of Work was officially approved and bearing in mind the structure of the Work Package, WP4 Data infrastructures prepared a work plan that was presented during the CHAIN-REDS Kick-off Meeting 5 for internal discussion. Such a work plan ought to address the following objectives already identified in the DoW: Obj1. Extend the CHAIN-REDS KB with Data Infrastructures. During the CHAIN project lifetime, the consortium made a big effort in implementing a KB that contained detailed information about the different Grid initiatives around the world. This information could be displayed in the CHAIN webpage in a three-fold basis: Country, DCI Site and DCI Table views. In the former, the user could search on a worldwide map his/her country of interest and click on it, so information about its domestic Grid status was displayed. In the DCIs views, information about the Distributed Computing Infrastructures per site geographically displayed on a map or per country showed in a table could be retrieved. For CHAIN- REDS, the KB is being enhanced with information related to Data infrastructures (see Information gathering and actions performed Section below for a deeper explanation). Obj2. Support the study of data infrastructures for a few VRCs. Aforementioned use cases should count on a community of users (VRCs)

9 CHAIN-REDS Project - Deliverable D4.1 Page #9 who would test and give feedback about the solutions related to interoperability aspects of data infrastructures proposed by CHAIN-REDS. In this sense, the project will support these VRCs in the processing of learning (if any), implementation (if any) and final validation. Obj3. Promote trust building towards open scientific data infrastructures across the world regions. Among the aspects referred in the previous item, it is of outmost importance trust building; by providing so, users from different groups will be able to access data previously stored by different researchers in different world regions relying not only on their accuracy, but also on their computational format, i.e. these users are retrieving the information they are looking for. Obj4. Study the opportunities of data sharing across different e- Infrastructures and continents. For achieving such a goal, it is necessary to survey different sources of information in order to look for synergies that could be exploited: groups of researchers (VRCs) who could make use of big amount of data in the countries belonging to the continents targeted by CHAIN-REDS; DCIs who could report on the users that usually transfer big amount of data too; and, on-going well-established data initiatives. Obj5. Provide proof-of principle use-cases for Data sharing across the continents. Select among the identified candidates, those who present promising opportunities for data sharing in terms of some standards and good practises selected by CHAIN-REDS. By doing so, it is expected to achieve a major and final goal, i.e. propose at least two use cases, which show common format for the stored data by means of software developments that could make all of them compatible and interoperable. Such a selection of use cases should be also endorsed by means of their respective MoUs with their representative communities, the action of which corresponds to WP4 milestone MS6 (to be fulfilled by Sep 2013). Then, this is the internal road-map of actions that have been planned in WP4: Action1. Analyse which standards are most used and, as a consequence, extended worldwide and accordingly propose best-practices to the communities to be approached, i.e. achieve Obj3 Action2. Define in what terms the CHAIN-REDS KB should be improved in order to firstly collect and lately display, in a dynamic way, information related to data infrastructures (Obj1) Action3. Analyse which repositories and what scientific fields related with are of interest to CHAIN-REDS according to Action1 and Action2 (Obj2 & Obj4) Action4. Survey the different countries and DCIs of interest to CHAIN-REDS and the well-established data-related initiatives in order to get their feedback about their usage of the identified standards and the computational platforms that are being utilised (Action2) and about the groups that are profiting from a huge employment of data and belong to the identified scientific fields (Action3). In addition, propose to the latter the CHAIN-REDS support for adopting the proposed standards (Obj2-Obj4) Action5. Analyse the results of the survey (Action4) and focus the WP4 efforts on two or three scientific communities in order to efficiently perform 9

10 CHAIN-REDS Project - Deliverable D4.1 Page #10 data sharing in terms of some standards and good practises (Obj5). This Action should count on the support from CHAIN-REDS in order to implement or adopt some standards if necessary, the collaboration of the identified communities by means of MoUs which would rule the scientific liaison and, the support of DCIs for making the real demonstration of data interoperability In the following Sections, information about the work done up to month 6 in the previous detailed road-map is described. 3. Standards adopted In order to work on any action or goal, the first key point is the definition of the way in which it is expected to be achieved. In the tasks related to WP4, this means the selection of the standards that will be fostered by CHAIN-REDS and, hopefully, adopted (if necessary) by the collaborative VRCs. At the same time, it is of outmost importance that such standards have a wide presence worldwide because of the intercontinental scope of CHAIN-REDS. Thus, after a deep analysis on the best practices that on-going data initiatives were carrying out, the following standards were selected by the project for pursuing trust building: Std1. OAI-PMH 6 for metadata retrieval. The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a low-barrier mechanism for repository interoperability. The service works in a way that data providers are repositories that expose structured metadata via OAI-PMH and service providers then make OAI-PMH service requests to harvest that metadata. OAI-PMH is a set of six verbs or services that are invoked within HTTP. Std2. Dublin Core 7 as metadata schema. Specifically, the Dublin Core Metadata Initiative (DCMI) is an open organization that has defined a set of vocabulary terms which can be used to describe resources for the purposes of discovery. The terms can be used to describe a full range of web resources, physical resources and objects. The original set of classic metadata terms, known as the Dublin Core Metadata Element Set, counted on 15 entries. Std3. SPARQL 8 for semantic web search. Resource Description Framework (RDF) is a standard model for data interchange on the Web. It is a directed, labelled graph data format for representing information in the Web. This specification defines the syntax and semantics of the SPARQL query language for RDF. SPARQL can be used to express queries across diverse data sources, whether the data is stored natively as RDF or viewed as RDF via middleware. SPARQL contains capabilities for querying required and optional graph patterns along with their conjunctions and disjunctions. SPARQL also supports extensible value

11 CHAIN-REDS Project - Deliverable D4.1 Page #11 testing and constraining queries by source RDF graph. The results of SPARQL queries can be results sets or RDF graphs Std4. XML 9 as potential standard for the interchange of data represented as a set of tables. Extensible Markup Language (XML) is a simple, very flexible text format derived from SGML (ISO 8879). Originally designed to meet the challenges of large-scale electronic publishing, XML is also playing an increasingly important role in the exchange of a wide variety of data on the Web and elsewhere. With this selection in mind, a work plan for improving the CHAIN-REDS KB by adding new functionalities was also planned. Both actions, i.e. the selection of standards and the work plan for the KB, were decided under the coordination of the CHAIN-REDS Technical Coordinator. In this way, Prof. Roberto Barbera s and WP4 teams have closely collaborated. The work plan is roughly described by the following steps: Integration of the KLIOS 10 services inside the KB in order to extend its functionalities. Knowledge Linking and sharing in research domains (KLIOS) is a project for developing small research projects and implement them in real-life use cases related to data sharing by means of metadata harvesting Dynamically include in the KB Data Repositories (DRs) and Open Access Data Repositories (OADRs) worldwide by using the already defined standards. The information related to these repositories should be presented to the user in the same way as the already available DCIs one. Take this integration as a proof-of-principle for demonstrating the work carried out. Repositories containing documents (articles, proceedings, books, etc.) were selected in principle for testing and depurating the KB functionalities with the aim of improving its characteristics to other fields and data (see items below) Provide the KB with the following capabilities, so further extraction and exploitation of raw data by any user could be performed o Semantic web enrichment o Semantic search engine o A tool for extracting the data associated to the repositories 4. Information gathering and actions performed Once that the standards were identified and the process for improving the KB was defined, information about the repositories of interest in the different regions targeted by CHAIN-REDS should be gathered. In addition, the implementation of the new KB capabilities should be launched off Information gathering For the first point, being CHAIN-REDS a second phase of the CHAIN project, where a big survey about DCIs was done, it was logical to profit from previous results and, mainly, from the list of national contacts, for submitting a short questionnaire about data repositories and adoption of standards

12 CHAIN-REDS Project - Deliverable D4.1 Page #12 In order to attract as many national coordinators as possible and get their feedback, a lightweight questionnaire was drafted by WP4 and lately circulated within the CHAIN-REDS Project Meeting Board in order to get consensus on a final version to be delivered. Its aim was to act as an entry point for the country, i.e. obtain the coordinates of researchers, engineers and personnel directly working on or with data repositories. The questionnaire can be found in Table I. Dear National representative, CHAIN-REDS project ( aims to promote Data Infrastructures among other several objectives. In this sense, it is either interesting in promoting Virtual Research Communities and Data repositories; thus, you can visit the CHAIN-REDS Applications ( and the Knowledge Base ( webpages, where you can find information about applications and Distributed Computing Infrastructures and Open Access Data Repositories respectively. It would be an outstanding asset for us if you could answer us three simple questions: 1.- Are you aware of any Open Access Data Repository in your country? If so, could you please forward us its contact person and/or its webpage? 2.- Are you aware of any other important Data repository in your country (or of interest to your colleagues) even if it is not an open one? 3.- Could you please send us the coordinates of groups or communities using data in the following fields? Agriculture e-government Earth Science Astronomy and Astrophysics Looking forward to hearing from you soon, kind regards, Rafael Mayo-Garcia (on behalf of the CHAIN-REDS project) Fig. 1 WP4 questionnaire delivered during the first quarter of the project lifetime The questionnaire was submitted during February and March 2013 to the national representatives of the following regions: Africa; Asia-Pacific; Asia-New Zealand; Central Asia; China; India; Latin America; Mediterranean and Middle East. It should be pointed out that a second row of questions arose when the first answers were received and a further link to a DR contact was established; this second iteration mainly consisted on gathering information about such repositories and their adoption of the standards identified by CHAIN-REDS (Std1- Std4), offering also specific support from the project to implement them if necessary. A lot of information was exchanged with many countries; the results can be summarised as follows. A cross-check process was established once the different national representatives 12

13 CHAIN-REDS Project - Deliverable D4.1 Page #13 gave their feedback. The reason for that was not to duplicate the information already contained in the CHAIN-REDS KB (see below) due to many repositories were reported per country. To those new, further contacts were set up for including these repositories either in the OADRs or the DRs views. Support from Prof. Roberto Barbera s team and WP4 was also offered in order to achieve as soon as possible the adoption of the standards promoted by CHAIN-REDS. New specific repositories (not previously included in the KB) provided by the national representatives referred to: a thesis repository in Malaysia 11 ; a seismic data repository 12 and an Earth science group in Morocco; the Scientific Data Center of CNIC 13 in China; e-government 14, agriculture 15, Earth science 16 and Life Science 17 repositories in Singapore; the Earth science data 18 in Namibia; e- government 19, 20, 21, 22, agriculture 23 and bioinformatics 24 repositories in South Africa; agriculture 25 and thesis 26 repositories in Mozambique; the multidisciplinary data centre maintained by the Laboratorio Nacional de Cómputaçao Científica 27 in Brazil; and an e-government repository 28 in Nigeria. In addition, CHAIN-REDS was aware of new initiatives by means of this survey. eifl 29, devoted to the collaboration with libraries in more than 60 developing and transition countries in Africa, Asia, Europe, and Latin America, was one of them. eifl reported on DataCite 30 (devoted to storing documents), Repositorio de Datos Científicos 31 (scientific data), Arhiv Druzboslovnih Podatkov 32 (social sciences), SERSCIDA 33 (social sciences) and National Institutes of Health data sharing 34 (biomedicine). It is worth mentioning the Data Intensive Research Infrastructure of South Africa (Dirisa), promoted by the Centre for High Performance Computing 35, the Open Data in Developing Countries 36 multi-country, multi-year Life Science BioMirror Resource, A*STAR

14 CHAIN-REDS Project - Deliverable D4.1 Page #14 study led by the World Wide Web Foundation, the Open Data for Development in Latin America and the Caribbean initiative 37, the UNESCO-HP Brain Gain Initiative 38 and the Particle Physics related project SCOAP3 39, which are initiatives that are of interest to CHAIN-REDS too. Also, the project has searched for projects interested in data processing in the meantime. The main one could be EUDAT 40. EUDAT aims to address the specific challenges of data management and exploit new opportunities using its vision of a Collaborative Data Infrastructure. Another important project related to data is OpenAIRE 41, which promotes an open access infrastructure for research in Europe. In addition, it could be mentioned that the Astrophysics community has a long track record in collaborative scenarios. The construction of large telescopes and facilities, which are usually built by international consortiums (see ESO 42, Pierre Auger 43, e-vlbi 44, etc.), has derived in the necessity of storing and accessing the data that they are continuously measuring in a seamless way, so any researcher worldwide could exploit them. Thus, the use of standards and common practices has been a must. In this sense, initiatives such as the International Virtual Observatory Alliance 45 (IVOA) are aligned with the CHAIN-REDS WP4 objectives. Last but not least, some reports coming from other sources have been consulted in order to prepare a good further impact of the CHAINREDS KB characteristics. Thus, Riding the wave - How Europe can gain from the rising tide of scientific data, a report of the High Level Expert Group on Scientific Data submitted to the European Commission, e-irg White Paper 2011, which contains a specific chapter for Data infrastructures, and Science as an open enterprise, published by the Royal Society, have been consulted. Several contacts with all of these initiatives have been established and it is expected to enhance the collaboration with them throughout the CHAIN-REDS lifetime beyond the incorporation of their repositories in the KB. After the end of the questionnaire delivery and retrieval of information, an analysis of the next steps to be taken was performed. Such a work was done on the basis of the identified scientific fields that would be mainly supported by CHAIN-REDS. These are Documents repositories (mainly article, papers, proceedings, etc.) Agriculture e-government 37 OD4D,

15 CHAIN-REDS Project - Deliverable D4.1 Page #15 Earth Science Astronomy and Astrophysics The selection of these areas was due to the fact that most of them already count on a huge amount of data which rely on the standards selected by CHAIN-REDS. In this way, the project objectives previously described will be better achieved and, hopefully, will have a higher impact in their related communities of users The CHAIN-REDS Knowledge Base As the survey was going on, a work on the development of new functionalities in the CHAIN-REDS KB 4 was performed. It should be noticed here that the KB was implemented considering e-infrastructures as an environment where research resources (hardware, software and content) can be readily shared and accessed where necessary to promote better and more effective research. Then, such environments integrate hard-, soft- and middleware components, networks, data repositories, and all sorts of support enabling virtual research collaborations with a final goal: to allow scientists across the world to do better (and faster) research using DCIs, independently of where they are and of the paradigm(s) adopted to build them. Thus, to better fulfil this long term milestone, the use of standards is more than an asset and, at the same time, a step forward to achieve sustainability. The first release of the CHAIN KB presented integrated dynamically updated information about DCIs. Information about Regional and National Research and Education Networks, National Grid Initiatives, Certification Authorities, Identity Federation Providers, Regional Operation Centres, Grid sites and Applications (and already running on a Science Gateway 46 ) was available in both Country and Table views. All this work was performed bearing in mind the concepts mentioned in the previous paragraph. But even when this on-line service was a clear step forward in harmonising the different regional infrastructure information, new capabilities should be incorporated in order to be aligned with CHAIN-REDS objectives (see Obj1-Obj5). Thus, it was decided to work with article and papers repositories on a first stage. Once standards (Std1-Std4) were identified to easily gather and access both OADRs and DRs, a demonstrator was built with them to visualise and access the repositories by means of both geo- and tab-views (as it was previously made for DCIs). Such a demonstrator was implemented with the advances carried out within the Knowledge Linking and sharing in research domains 10 (KLIOS) project. KLIOS is based on the interconnection and the integration of scientific resources through a grid of meta-data network and provide the following services: metadata harvesting; semantic enrichment; and, linked data semantic search. Basically, the new KB capability is composed of a multi-layer structure where two harvesters running on either Grid or Cloud search for OAI-PMH 6 end-points from OADRs and DRs. Above them, a semantic web-enrichment layer is used to act as a previous step before the linked-data search engine, which is on the top

16 CHAIN-REDS Project - Deliverable D4.1 Page #16 Linked Data Search Engine Triples Store Semantic Web Enrichment Harvester GRID Harvester Cloud OADR DR Fig. 2 KLIOS Schema The process of the metadata harvesters is as follows: Get the address of each repository publishing an OAI-PMH standard endpoint Retrieve, using the OAI-PMH repository address, the related Dublin Core 7 encoded metadata in XML 9 format Get the records from the XML files and, using the Apache Jena API 47, transform the metadata in RDF 48 format Save the RDF files into a Virtuoso 49 triple store according to an OWLcompliant ontology built using Protégé 50. CHAIN-REDS KB reports of 2,488 OADRs and 507 DRs (Apr, 18 th 2013). From these repositories, the following information is depicted: the country where the data is stored; the name of the repository; the scientific domain it belongs to; and, the organisation is maintaining it. All this information can be also geographically displayed on a world map. OADRs counts on DRIVER 51, OpenAIRE 41 and OpenDOAR 52 and currently refers to more than 30 million documents and, regarding DRs, they include Databib 53 and DataCite

17 CHAIN-REDS Project - Deliverable D4.1 Page #17 Fig. 3 Snapshot of the OARDs Site View of the CHAIN-REDS Knowledge Base As it has been aforementioned, as a legacy from CHAIN, the KB has 86 entries in its DCIs Table view, which reports on the country where the DCI is settled, the regional network, the National Research Education Network (NREN) and the National Grid Initiative (NGI) it belongs to, the Certification Authority (CA) and the Identity Federation (IdF) it relies on for accessing it, the Regional Operation Centre (ROC) it is connected to and the sites it counts on. As in the previous case, the same information can be showed searching on a world map. 17

18 CHAIN-REDS Project - Deliverable D4.1 Page #18 Fig. 4 Snapshot of the DCIs Country View of the CHAIN-REDS Knowledge Base The next action was to correlate OADRs and DRs to create linked data and discover new knowledge through semantic enrichment of metadata. Thus, a user can now access the Linked Data tab 54 in the CHAIN-REDS website, select (if desired) which language will use for its search, enter (if desired) a free keyword or some from a list of them and final make the semantic search. The results are displayed on the screen on a two-fold possibility: as a list of them or as a graph where filters for visualisation can be applied in order to detect semantic links. Doing so, new information can be obtained from the graphs, i.e. new knowledge through semantic enrichment of metadata. On April 2013, the CHAIN-REDS KB counted on data repositories storing articles and papers; then, by performing a semantic search with the Linked Data tool, authors who work on a scientific topic, articles related to the same kind of research or investigations carried out with the same kind of facility or experimental setup, for example, can be easily discovered

19 CHAIN-REDS Project - Deliverable D4.1 Page #19 Fig. 5 Snapshot of a Linked Data search in the CHAIN-REDS Knowledge Base The next steps will be to test and make as robust as possible the current KB capabilities and, later on, to implement a tool for extracting the data associated to the OADRs. This is an interesting challenge to uniquely correlate (scientific) data appearing in papers with applications/codes that could be used to analyse them and vice versa, i.e. it would be possible to go across the knowledge path both ways. 5. Identified Communities This section describes not only the fields, but the specific initiatives that have 19

20 CHAIN-REDS Project - Deliverable D4.1 Page #20 been identified by CHAIN-REDS for collaborating with them in the achievement of the pledged objectives. In other words, once that the KB is being implemented by the use of article repositories (which cover almost every scientific, social and cultural discipline) as a very useful and productive test-bed, specific actions are needed for identifying those two use cases that will be presented as a proof-ofconcept for evaluating the work carried out by CHAIN-REDS WP4 (Obj5) Agriculture aginfra One of the main promising fields in data and metadata management is agriculture. This is so because such a field is an ancient tradition worldwide i.e. almost every country has relied on agriculture since many years ago and currently counts on services related to it, which is a key point in CHAIN-REDS due to its presence in many Continents. Thus, a metadata framework has even been introduced by the Food and Agriculture Organization of the United Nations (FAO). It was recognized that there was a strong need for statistical metadata, which would provide better understanding of all the data items and the way to obtain them within the national system of agricultural statistics. The idea in the agricultural field is to establish metadata databases for food and agricultural statistics due to it is considered as one of the key components for improving data quality and statistical development. The concept of metadata here describes all aspects of the national systems of agricultural statistics on how, when, where, why, and by whom the data are collected. Thus, metadata is taken as a primary tool in describing and managing information resources, and also as a useful and beneficial process to both users and producers of statistics. The challenge faced by the management of metadata at the international level is how to design a framework so it can be used by countries to collect the relevant and succinct information in a manageable and comparable way. Such information would refer to the current stage of the national agricultural statistics for assessing data quality, identifying areas of further development and assisting to plan, design, implement, and coordinate national and regional statistical capacity building programs and activities. In order to accomplish such a task in an affordable way, CHAIN-REDS has established a collaboration with the FP7 aginfra project 55, which is participated by an international institution (FAO) and partners from Europe, Asia and Latin America. aginfra aims to set up a data infrastructure to support agricultural scientific communities promoting data sharing and development of trust in agricultural sciences. It also plans to improve service deployment for data by transferring scientific and technological results from the agricultural field into real outcomes. Moreover aginfra is proposing to create a high interoperability between agricultural and other data resources. Thus, the project goals are aligned with the strategic initiative Coherence in Information for Agricultural Research for Development (CIARD) that mobilizes and supports institutions in making agricultural research results more accessible globally

21 CHAIN-REDS Project - Deliverable D4.1 Page #21 aginfra is targeting the integration of several domains that cover both areas of specific research focus (e.g. agricultural economics) and areas where a particular type of information provides a platform for research activity in general (e.g. bibliographic resources). The project has a clear intercontinental potential since it counts on an international partner as FAO is and also with European, Latin American and Asian partners. In addition, even when agriculture is a resource in almost every country in the world, from the e-infrastructure point of view, it is also a key point that aginfra is relying its computing power and administration on the same infrastructures targeted by CHAIN-REDS in WP3 and has also adopted the Science Gateway paradigm e-government ENGAGE It is well known that metadata is data about other data or objects, which are used to describe digitized and non-digitized resources located in a distributed system in a network environment. In e-government applications it may be used, amongst other for the discovery and retrieval of government information, as well as to assist in the management of government electronic resources. In other words, metadata is the key to interoperability, one of the major CHAIN-REDS general goals. Three characters of e-government which make e-government different compared to other applications, such as e-commerce, have been identified: access, structure and accountability. In e-government, public agencies are responsible for providing access to information and services for everyone living within a region, all of whom will have varying levels of IT skills including individuals with lower incomes and disabilities. This is why, organizing of e-government electronic collections on the internet in a way which help users to search and locate government information without needing details of government structure, or to find government services without knowing which agency delivers them, is a fundamental in e-government. Thus, metadata is a valuable tool in e- Government applications to make seamless flow of information and services across government and support citizens finding government information and services more easily. CHAIN-REDS has identified the FP7 ENGAGE project 56 as an ideal initiative to collaborate with. ENGAGE is an infrastructure for open, linked governmental data provision towards research communities and citizens. Its main goal is the deployment and use of an advanced service infrastructure, incorporating distributed and diverse public sector information resources as well as data curation, semantic annotation and visualisation tools, capable of supporting scientific collaboration and governance related research from multi disciplinary scientific communities, while also empowering the deployment of open governmental data towards citizens

22 CHAIN-REDS Project - Deliverable D4.1 Page #22 The ENGAGE e infrastructure is envisaged to promote a highly synergetic approach to governance research, by providing the ground for experimentation to actors from both ICT and non ICT related disciplines and scientific communities, as well as by ensuring that the scientific outcomes are made accessible to the citizens, so that they can monitor public service delivery and influence the decision making process. ENGAGE will provide enhanced services in the data e- Infrastructure layer while on the other hand building a community that can exploit the e Infrastructure services. The project counts on a platform in a beta status 57, which can be a good point for collaboration between this initiative and the groups interested in e-government approached worldwide by means of the CHAIN-REDS WP4 survey. In this sense, also the CHAIN-REDS KB could be provided as an entry point for a closer liaison Earth Sciences EarthServer With explosively increasing volumes of remote sensing, models and other Earth Science data available and the popularity of the Internet, Earth scientists are now facing challenges to publish and to find interesting data sets effectively and efficiently. This is, one of the main barriers to exploiting the great wealth of global Earth science data available today is that researchers are unable to rapidly search and find data relevant to their studies. This data is spread across a large number of archives maintained by different institutions employing a bewildering array of different data description languages. Metadata have been recognized as a key technology to ease the search and retrieval of Earth science data. Metadata is used in all aspects of Earth Science data lifecycle from the initial measurement gathering to the accessing of data products. Research campaigns use metadata in their science data products when describing information such as the instrument/sensor, operational plan, and geographically region. Acting as the curator of the data products, data centres employ metadata for preservation, access and manipulation of data. For exploring the CHAIN-REDS objectives in this field, the project is collaborating with the FP7 initiative EarthServer 58, which is working on establishing open access and ad-hoc analytics on several extreme-size Earth Science data related to cryospheric, airborne, atmospheric and planetary sciences and also to geology and oceanography. Among its main innovations, it is worth mentioning its goals for: integrating coverage, feature, and metadata queries, including all Open Geospatial Consortium (OGC) coverage types; performing transparent queries over heterogeneous file archives and databases; paving the way for Petabyte services (cloud distribution, parallelization, supercomputers); and, providing comprehensive OGC standards support for coverage data and services. EarthServer, counts on a Science Gateway based on that developed by CHAIN

23 CHAIN-REDS Project - Deliverable D4.1 Page #23 REDS project, so further collaboration is expected between this two projects in, at least, one of the EarthServer analytics in order to build trust building proof-ofprinciple. To achieve such a goal, it is also worth mentioning that both projects share the same e-infrastructures and that the data collected in EarthServer are of interest in any regions of the world Astrophysics IVOA As in the previous cases, Astrophysics and Astronomy are fields where data has been collected from many years ago, producing in this way a heterogeneous set of information. In addition, it should be pointed out that not only raw data storage coming from astronomical measurements or calculations, but also this kind of data that can be extracted from images and the images themselves are objects that require a metadata scheme. In this community, neither the international collaborations supporting big facilities nor the bureaus or societies dictate how a data centre handles its own archive. However, a Virtual Observatory-layer is needed to translate any locally stored data to an agreed standard. Data providers are then advised to systematically collect metadata about the curation process, assign unique identifiers, describe the general content of a collection, and provide interface and capability parameters of services. In the context of Astrophysics, CHAIN-REDS has identified the International Virtual Observatory Alliance 45 (IVOA) as an ideal collaborator. The Virtual Observatory (VO) is the vision that astronomical datasets and other resources should work as a seamless whole. Many projects and data centres worldwide are working towards this goal. IVOA is an organisation that debates and agrees the technical standards that are needed to make the VO possible. It also acts as a focus for VO aspirations, a framework for discussing and sharing VO ideas and technology, and a body for promoting and publicising the VO. Since its formation in 2002, the IVOA has been working on reaching truly worldwide cohesion in debating and agreeing key astronomical standards (it is endorsed by the International Astronomical Union as the accepted method for producing astronomical data-related standards), establishing a forum for discussing and debating astronomical data technology in general as well VO standards in particular and, achieving rapid agreement on an initial set of basic standards (a table exchange format, a specification for simple catalogue and image query services, the definition of metadata describing resources, a dictionary for standardised column names, and a suite of standards allowing the construction of VO registries). The IVOA is also pursuing on providing further standards, including those needed for virtual storage addressing, single sign on, semantic reasoning, grid and web service modularisation. It counts on a Grid & Web Services Working Group, which has developed and interface to access both PRACE and EGI infrastructures for the scientific computations, which are targeted e-infrastructures in CHAIN-REDS. Even more, it proposes services closely related to those proposed by EUDAT (see below), which are also the ones proposed by CHAIN-REDS. 23

24 CHAIN-REDS Project - Deliverable D4.1 Page # General data related initiatives EUDAT and documents repositories As it has been aforementioned, the first kind of repositories that CHAIN-REDS has been working with has been document repositories. Thus, articles, papers and proceedings have been included in the project KB by means of the OADRs and DRs. This action has been of outmost importance because it has allowed the project to develop semantic methodologies for improving the retrieval of data and related information and, also, plan a major challenge as the extraction of raw data from articles is. But in addition, CHAIN-REDS has identified some general data related initiatives that represent major actors in the European context. Among them, the main one could probably be EUDAT 40. This initiative was created due to the increase of communication networks, distributed grids and HPC facilities, which has provided European researchers from all fields with state-of-the-art instruments and services that support the deployment of new research facilities on a pan-european level. As a consequence, an accelerated proliferation of data (newly available from powerful scientific instruments, simulations and digitization of library resources) has aroused, which has created a new impetus for increasing efforts and investments in order to tackle the specific challenges of data management, and to ensure a coherent approach to research data access and preservation. EUDAT aims to address these challenges and exploit new opportunities using its vision of a Collaborative Data Infrastructure. A first interaction between EUDAT and CHAIN-REDS has been established, where the commonalities of both projects have been detailed and the CHAIN-REDS KB has been proposed as an example of data management and standards adoption by means of the OADRs and DRs views. Conclusions CHAIN-REDS started on Dec, 1 st 2012 focused on promoting and supporting technological and scientific collaboration across different e-infrastructures established and operated in various continents. A major step forward in this direction is to promote instruments and practices that can facilitate their inclusion in the community of users, i.e. the use of standards. Addressing basic issues such as data persistency, accessibility and interoperability is the related goal for WP4 Data infrastructures. During the first semester of the project, WP4 has defined a work plan where the objectives are set and, according to them, a list of consecutive actions has been planned. With regards to the standards adoption in the field of data preservation, access and use, four (4) have been selected. Bearing them in mind, the CHAIN-REDS Knowledge Base is being improved. Now, it includes DCIs information, as in CHAIN project lifetime, but also OADRs and DRs. These repositories can be 24