GEOSS Common Infrastructure (GCI) Research Engineering Report GEO Architecture Implementation Pilot, Phase 5 GEOSS Architecture Implementation Pilot

Transcription

1 GEOSS Common Infrastructure (GCI) Research Engineering Report GEO Architecture Implementation Pilot, Phase 5 GEOSS Architecture Implementation Pilot Version 1.1 Content developed by the GEO Architecture Implementation Pilot Licensed under a Creative Commons Attribution 3.0 License

2 Revision History Version Date Editor and Content providers Comments /02/2013 Mattia Santoro, Stefano Nativi Draft /02/2013 Mattia Santoro Draft /02/2013 Stefano Nativi Draft /02/2013 Lionel Menard Draft /02/2013 Mattia Santoro Added Test Contributors Table Document Contact Information If you have questions or comments regarding this document, you can contact: Name Organization Contact Information Stefano Nativi CNR-IIA Mattia Santoro CNR-IIA Max Craglia EC-JRC Test Contributors Date Access Discovery, Helper Apps Discovery, Access Editor and Content providers Perry Peterson, Idan Shatz Mines Paris-Tech Technical Universitat Dresden/GLUES Comments Puxis Innovation Lionel Menard Matthias Müller, Christin Henzen Discovery Compusult Craig Coffin Page 2

3 Table of Contents 1. Introduction Scope of this document GEOSS AIP Summary of the Pilot development Future work 4 2. Pilot Objectives 4 3. Scenario: Test new Client/Server interoperability with GEO DAB Actors Scenario Events Post-Conditions Special Requirements 7 4. Enhance the GEO Discovery & Access Broker Access Protocol Queryable Extending the ISO Queryables Helper Applications Enriching Metadata Helper Application Registry Helper Application with Context How to provide information Testing Helper Application References Annex 1 Query by Access Protocol examples 15 Page 3

4 GEOSS Common Infrastructure (GCI) Research 1. Introduction 1.1 Scope of this document This document describes the activity which was done by the GEOSS Common Infrastructure (GCI) Research WG of the GEOSS Architecture Implementation Pilot (Phase 5). 1.2 GEOSS AIP In general, GEOSS Architecture Implementation Pilot (AIP) task develops process and infrastructure components for the GCI and the broader GEOSS architecture as a means of coordinating cross-disciplinary interoperability deployment. The AIP Task provides phased delivery of components to GEOSS operations, with each phase consisting of: architecture refinement based on user interactions; component deployment and interoperability testing; and SBA-focused demonstrations. This Engineering Report (ER) is a key result of the fifth phase of AIP. AIP-5 was conducted from January 2012 to December A separate ER describes the overall process and results of AIP-5 and thereby provides a context for this ER Summary of the Pilot development The main focus of this GCI Research activity was to test the use of and enhance the GEO Discovery and Access Broker (GEO DAB): the GCI middleware framework allowing discovery and access of heterogeneous resources from different information systems and capacities. The activity was structured according to a couple of tasks: 1. Test the GEO DAB accessibility and usability for new client/server systems/components; in particular: a. Provide GEO DAB capabilities and functionalities to existing clients/applications/community Portals; this allows them to make use of the GCI, and hence of GEOSS. b. Test the brokerage of new capacities and datasets. 2. Enhance the GEO Discovery & Access Broker a. To test new functionality and capabilities in order to support data access and use. The GEO Discovery and Access Broker (DAB) framework was developed by the EuroGEOSS 2 research programme of the European Commission under the leadership of the National Research Council of Italy (CNR) and of the European Commission s Joint Research Centre (JRC) (Vaccari et al., 2012). The GEO DAB is currently operated by the CNR with the support of the JRC (for the semantic assets and services). It is operated as part of the GEOSS Common Infrastructure (GCI) and provides intermediation services among heterogeneous capacities and information systems. The GEO DAB framework consists of: a Discovery Broker (Nativi and Bigagli, 2009) with an extension for semantic augmentation (Santoro et al., 2012) and an Access Broker. 1.4 Future work Both the testing and the development/consolidation of new functionalities are envisioned to continue in the next phase of GEOSS AIP (i.e. AIP-6). 2. Pilot Objectives The main objective of this activity was twofold: (a) to test GEO DAB entry-level interoperability for both Users (i.e. Community Portal/Clients) and Data Producers (i.e. existing capacities and systems); (b) to further develop GEO DAB in order to improve its functionalities as far as data access and use are concerned. 1 A listing of all AIP-5 Engineering Reports: Page 4

5 GEO Architecture Implementation Pilot, Phase 5 Version: 1.1 Date: 12/02/2013 GCI Research Engineering Report 3. Scenario: Test new Client/Server interoperability with GEO DAB This task aimed to test the GEO DAB with new servers providing catalog and/or access services to increase the number of resources available through the GCI. Figure 1 - GEO DAB Overview 3.1 Actors The testing phase involved the following actors: GEOSS Resource Providers: for the testing task they include data providers from different SBAs (e.g. energy, climate, etc.). They provide resources (data, metadata, services, clients, etc.). The following organizations provided servers for testing: o MINES ParisTech o o WebService-Energy catalog; Technical Universitat Dresden/GLUES TUD-GLUES THREDDS Data Server; TUD-GLUES Web Processing Service; Compusult Satellite Imagery Catalog; Page 5

6 o Pyxis Innovation WorldView Client Applicatin; GCI Operator: The GEO DAB operator o National Research Council of Italy (CNR) GEO DAB 3.2 Scenario Events Discovery Broker Test The test of the Discovery Broker with new servers was conducted as it follows: 1. The GEO DAB administrator deployed a separate test instance for the AIP-5; 2. The GEO DAB administrator configured the AIP-5 test instance to broker the new servers; 3. The server providers accessed the GEO DAB test client to discover their records and assess the validity of the discovered resources; 4. The server providers feedbacks were collected by the GEO DAB administrator to fix possible identified bugs. Table 1 lists the test results for the GEO Discovery Broker. Table 1 - GEO Discovery Broker Tests Service TUD- GLUES THREDDS Data Server TUD- GLUES Web Processing Service MINES ParisTech Web WebService- Energy Compusult Catalog Service Number of Resources/GEOSS Data Core 159/0 179/0 230/ /0 Comments Discovery of TDS Metadata: OK Missing time metadata in TDS, the broker might access the associated WMS capabilities for each TDS dataset in order to enrich the metadata with time information. Discovery of all Processes: OK Presentation of Process inputs: OK Discovery of Metadata: OK Data CORE and GEOSS NoMonetaryCharge discovery tags: OK Helper Application: OK Some issues were identified query by keyword and bounding box. Page 6

7 Access Broker Test The testing procedure described in the above section was applied to test the Access Broker as well. The test configuration is depicted in Figure 2. Two different clients were used to visualize data using the OGC WMS interface: The Pyxis Innovation WorldView Client; The GEO DAB test client. Data were retrieved from different sources: ehabitat WCS service TUD-GLUES THREDDS/OPeNDAP Figure 2 - Access Broker Test Configuration Both TUD-GLUES and ehabitat WCS data were correctly retrieved and transformed by the Access Broker. When using the GEO DAB test client, the returned data was shown shifted on one cell up and right. This bug was then fixed on the Access Broker side. Figure 3 depicts the Pyxis WorldView client access data using the Access Broker. 3.3 Post-Conditions NA 3.4 Special Requirements NA Page 7

8 Figure 3 - Pyxis Innovation WorldView using the GEO Access Broker Page 8

9 4. Enhance the GEO Discovery & Access Broker This task developed new capabilities and functionalities for the GEO DAB to further support users requirements. In particular, two enhancements were implemented and tested: 1. Support of a new queryeable element in the Discovery Broker: Access Protocol Queryable 2. Helper Application Support 4.1 Access Protocol Queryable This extension allows client applications to retrieve only datasets which are accessible through a particular access service type. This feature was tested by NOAA and is presently used by the OneGeology operational portal. The following sections provide details about how to query the GEO DAB with the new queryable Extending the ISO Queryables Consider the use case in which the user wants to filter the results in order to retrieve only datasets which are accessible through a WMS protocol. To be able to express this additional constraint (access via WMS) the Discovery Borker accepts an extended set of queryables. The new queryable element is encoded with the following qualified name essi:transferprotocol where essi is the xml name space corresponding to The fist prototype implementation of this extension allows to filter by OGC Access Service protocols. Thus, acceptable values for the above element are listed in Table 2. Table 2 - List of TransferProtocol Queryable Values Service Type Queriable Value OGC WCS OGC WMS OGC WFS urn:ogc:servicetype:webcoverageservice:1.0:http urn:ogc:servicetype:webcoverageservice:1.1:http urn:ogc:servicetype:webmapservice:1.1.1:http urn:ogc:servicetype:webmapservice:1.3.0:http urn:ogc:servicetype:webfeatureservice:1.0.0:http urn:ogc:servicetype:webfeatureservice:1.1.0:http OGC SOS urn:ogc:servicetype:sensorobservationservice:1.0.0:http Note: this first prototype does not take into account the version number Page 9

10 Annex I provides some query by access protocol examples. 4.2 Helper Applications Once the user finds a dataset of interest for her/his research, it is often the case that she/he is not provided with any tool which is able to access and exploit that dataset. In fact, assuming that the dataset is accessible on the Web, there exist several web service protocols which might be used to retrieve it as well as encoding formats for the dataset itself. A Helper Application is a client application which is able to access and/or visualize a dataset provided through: One or more web service protocols One or more encoding formats Enriching Metadata To provide users with a set of Helper Applications, it is needed to enrich providers metadata record with the needed information to access the data with one or more Helper Applications. Figure 4 depicts the sequence diagram describing the steps for the metadata enrichment. Essentially, for each discovered record the GEO DAB must: 1. Extract the access information for the dataset (that is, web service protocol, encoding format, and dataset name); 2. Query the Helper Application Registry (see Helper Application Registry section) to retrieve metadata of Helper Application which are able to access and use a dataset provided according to the access information identified in step 1; 3. Enrich the metadata record with the information to use the discovered Helper Applications. Moreover, when the discovered metadata record already provides a Helper Application, this should be marked as the provider s recommended Helper Application Helper Application Registry This registry provides descriptions of the available Helper Applications. These descriptions are used to execute steps 2 and 3 of the metadata enrichment phase described in the previous section. To populate the registry, a set of information is needed from Helper Application providers. In addition to the traditional information (title, abstract, keyword, etc.), Helper Application providers must provide the following information: 1. Supported Web Service protocols 2. Supported data encoding formats (preferably mime types) 3. URL of the Helper Application Page 10

11 Figure 4 - Enrich Metadata with Helper Applications, sequence diagram Helper Application with Context A Helper Application might be invoked with a context. That is, it might be launched with a link which already contains the link to the dataset to access. In this case, additional information must be provided to allow the GEO DAB to encode the context into the link to launch the Helper Application. The context is comprised of the following information (hereafter context items): 1. URL of the remote service providing the dataset 2. Name of the dataset on the remote service 3. Protocol of the remote service For each of the above items which is supported by a Helper Application, providers must document the syntax and encoding. That is, how to create a link which launches the Helper Application with a context. The GEO DAB will build links accordingly How to provide information Helper Application providers can provide the needed information in any form. It is up to the GEO DAB team to encode and publish such information, which will then be assessed by providers. Page 11

12 In summary the following set of information is needed: Table 3 - Helper Application Information Information Supported Web Service protocols Supported Data Encoding formats URL of Helper Application Syntax and encoding of the remote URL parameter Syntax and encoding of the parameter for the dataset name on the remote service Syntax and encoding of the parameter for the remote service protocol Description A list of access protocols which the Helper Application can use to access data A list of encoding formats (preferably expressed as mime- types) which the Helper Application can handle URL to access and/or download the Helper Application This is used to create a link which launches the Helper Application already connected to a remote access service This is used to create a link which launches the Helper Application already connected to a particular dataset of a remote access service This is used to create a link which launches the Helper Application already connected to a remote access service using a particular protocol O M M O O O Obligation (Mandatory/Optional) Testing Helper Application The experimentation of the Helper Application support was conducted in collaboration with the Energy WG (Menard et al., 2013), focusing on the recognition of provider recommended Helper Application. In fact, as described above, the catalog provides this information for datasets that can be exploited on WebGIS clients. The test was done using the GEO DAB test client. Figure 5 depicts a set of results from the MINES ParisTech WebService- Energy catalog ( and the associated Helper Application highlighted as a Provider Recommended Helper Application (i.e. in green). Instead, when a generic (i.e. not provider recommended) Helper Application is linked to a discovered dataset, this is shown as in Figure 6. Page 12

13 Figure 5 Provider Recommended Helper Application in the GEO DAB Figure 6 - Generic Helper Application in the GEO DAB 5. References Menard, L.; et al.; Energy Scenario Generation of local atlases for decision-support in solar energy policy planning and private investment Engineering Report GEO Architecture Implementation Pilot, Phase 5. GEOSS Architecture Implementation Pilot, 2013 Nativi, S.; Bigagli, L.; "Discovery, Mediation, and Access Services for Earth Observation Data" Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of, vol.2, no.4, pp , Dec. 2009, doi: /JSTARS Santoro, M.; Mazzetti, P.; Nativi, S.; Fugazza, C.; Granell, C.; Diaz, L.; Methodologies for Augmented Discovery of Geospatial Resources, in L. Díaz, C. Granell, & J. Huerta (Eds.), Discovery of Geospatial Resources: Methodologies, Technologies and, Emergent Applications, (pp ), doi: / ch009 Page 13

14 Vaccari, L.; Craglia, M.; Fugazza, C.; Nativi, S.; Santoro, M.; "Integrative Research: The EuroGEOSS Experience" Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of, vol.5, no.6, pp , Dec. 2012, doi: /JSTARS Page 14

15 6. Annex 1 Query by Access Protocol examples Below we list some CSW GetRecords queries which make use of the new queryable element. Note: to use the new queryable element, the xmlns:essi= must be added to the GetRecords request. All Resources Accessible through OGC WMS <?xml version="1.0" encoding="utf-8"?> <csw:getrecords xmlns:csw=" xmlns:ogc=" xmlns:gmd=" xmlns:apiso=" xmlns:ows=" xmlns:essi=" xmlns:xsd=" xmlns:gml=" xmlns:xsi=" service="csw" version="2.0.2" resulttype="results" outputformat="application/xml" xsi:schemalocation=" outputschema=" startposition="1" maxrecords="5"> <csw:query typenames="gmd:md_metadata"> <csw:elementsetname typenames="gmd:md_metadata">full</csw:elementsetname> <csw:constraint version="1.1.0"> <ogc:filter> <ogc:propertyisequalto> <ogc:propertyname>essi:transferprotocol</ogc:propertyname> <ogc:literal>urn:ogc:servicetype:webmapservice:1.1.1:http</ogc:literal> </ogc:propertyisequalto> </ogc:filter> </csw:constraint> </csw:query> </csw:getrecords> All Resources Accessible through OGC WMS AND Matching the Keyword Water <?xml version="1.0" encoding="utf-8"?> <csw:getrecords xmlns:csw=" xmlns:ogc=" xmlns:gmd=" xmlns:apiso=" xmlns:ows=" xmlns:essi=" xmlns:xsd=" xmlns:gml=" xmlns:xsi=" service="csw" version="2.0.2" resulttype="results" outputformat="application/xml" xsi:schemalocation=" outputschema=" startposition="1" maxrecords="5"> <csw:query typenames="gmd:md_metadata"> <csw:elementsetname typenames="gmd:md_metadata">full</csw:elementsetname> <csw:constraint version="1.1.0"> <ogc:filter> <ogc:and> <ogc:propertyisequalto> <ogc:propertyname>essi:transferprotocol</ogc:propertyname> <ogc:literal>urn:ogc:servicetype:webmapservice:1.1.1:http</ogc:literal> </ogc:propertyisequalto> <ogc:propertyislike wildcard="*" singlechar="#" escapechar="!"> Page 15

16 </ogc:and> </ogc:filter> </csw:constraint> </csw:query> </csw:getrecords> <ogc:propertyname>apiso:anytext</ogc:propertyname> <ogc:literal>*water*</ogc:literal> </ogc:propertyislike> All Resources Accessible through (OGC WMS OR OGC SOS) AND Matching the Keyword Water <?xml version="1.0" encoding="utf-8"?> <csw:getrecords xmlns:csw=" xmlns:ogc=" xmlns:gmd=" xmlns:apiso=" xmlns:ows=" xmlns:essi=" xmlns:xsd=" xmlns:gml=" xmlns:xsi=" service="csw" version="2.0.2" resulttype="results" outputformat="application/xml" xsi:schemalocation=" outputschema=" startposition="1" maxrecords="5"> <csw:query typenames="gmd:md_metadata"> <csw:elementsetname typenames="gmd:md_metadata">full</csw:elementsetname> <csw:constraint version="1.1.0"> <ogc:filter> <ogc:and> <ogc:or> <ogc:propertyisequalto> <ogc:propertyname>essi:transferprotocol</ogc:propertyname> <ogc:literal>urn:ogc:servicetype:webmapservice:1.1.1:http</ogc:literal> </ogc:propertyisequalto> <ogc:propertyisequalto> <ogc:propertyname>essi:transferprotocol</ogc:propertyname> <ogc:literal>urn:ogc:servicetype:sensorobservationservice:1.0.0:http</ogc:literal> </ogc:propertyisequalto> </ogc:or> <ogc:propertyislike wildcard="*" singlechar="#" escapechar="!"> <ogc:propertyname>apiso:anytext</ogc:propertyname> <ogc:literal>*water*</ogc:literal> </ogc:propertyislike> </ogc:and> </ogc:filter> </csw:constraint> </csw:query> </csw:getrecords> All Resources Accessible through OGC SOS AND Overlapping the BoundingBox [E: ; S: 4.801; W: ; N: ] Page 16

17 <?xml version="1.0" encoding="utf-8"?> <csw:getrecords xmlns:csw=" xmlns:ogc=" xmlns:gmd=" xmlns:apiso=" xmlns:ows=" xmlns:essi=" xmlns:xsd=" xmlns:gml=" xmlns:xsi=" service="csw" version="2.0.2" resulttype="results" outputformat="application/xml" xsi:schemalocation=" outputschema=" startposition="1" maxrecords="5"> <csw:query typenames="gmd:md_metadata"> <csw:elementsetname typenames="gmd:md_metadata">full</csw:elementsetname> <csw:constraint version="1.1.0"> <ogc:filter> <ogc:and> <ogc:propertyisequalto> <ogc:propertyname>essi:transferprotocol</ogc:propertyname> <ogc:literal>urn:ogc:servicetype:sensorobservationservice:1.0.0:http</ogc:literal> </ogc:propertyisequalto> <ogc:bbox> <ogc:propertyname>apiso:boundingbox</ogc:propertyname> <gml:envelope srsname="epsg:4326"> <gml:lowercorner> </gml:lowerCorner> <gml:uppercorner> </gml:upperCorner> </gml:envelope> </ogc:bbox> </ogc:and> </ogc:filter> </csw:constraint> </csw:query> </csw:getrecords> Page 17