Download Service Implementing Rule and Technical Guidance

Transcription

1 Download and Transformation Draft Implementing Rules Presentation for INSPIRE Initiatives Download Service Implementing Rule and Technical Guidance Olaf Østensen Statens kartverk Norwegian Mapping Authority

2 Subject

3 General remarks to comments received NS DT appreciates all comments Comments have been very helpful in improving the document significantly Some comments have not been accepted Does not mean they are necessarily wrong Can be judged to be out of scope of the IR, or Previous decisions have been different, but have prejudice

4 Summary 24 organizations have commented 392 comments 186 technical comments 100 editorial comments 102 general comments 4 other

5 392 comments 260 accepted 91 not accepted 41 other action Summary draft resolutions

6 Summary per organization technical editorial general other Ministère de l'écologie, de l'énergie, du développement durable et.. SITAD SP Sistema Informativo Territoriale e Ambientale Diffuso... Lenkungsgremium GDI DE

7 Summary draft resolution Accepted Not accepted Other Ministère de l'écologie, de l'énergie, du développement durable et.. SITAD SP Sistema Informativo Territoriale e Ambientale Diffuso... Lenkungsgremium GDI DE

8 Summary draft resolution Accepted Not accepted Other Ministère de l'écologie, de l'énergie, du développement durable et.. SITAD SP Sistema Informativo Territoriale e Ambientale Diffuso... Lenkungsgremium GDI DE

9 Summary IGN B Ministère de l'écologie, de l'énergie, du développement durable et.. SITAD SP Sistema Informativo Territoriale e Ambientale Diffuso... Accepted Not accepted Other Lenkungsgremium GDI DE

10 Actions taken Smaller editorial changes (e.g. misspellings) Terminology and concepts harmonised (e.g. with INSPIRE Glossary) Sections, and text in general, rewritten to be more clear and distinct Some sections and paragraphs deleted, for instance because they did not add value or were not suitable in an IR Requirements relieved by making mandatory requirements optional (TG)

11 Implementing Rule for Download services Article Member States shall establish and operate a network of the following services for the spatial data sets and services for which metadata have been created in accordance with this Directive: (a) (c) download services, enabling copies of spatial data sets, or parts of such sets, to be downloaded and, where practicable, accessed directly; (d)

12 The INSPIRE network service architecture Data Service Appl. Tiers Registry Service Register Data Discovery Service Service Metadata Applications and Geoportals Data Set Metadata RM Services RM Data View Service Download Service Spatial Data Set Transf. Service Rights Management Layer InvokeSD Service Thematic DS Framework for harmonized DS

13 Directive text 1. Member States shall establish and operate a network of the following services for the spatial data sets and services for which metadata have been created in accordance with this Directive: (a) (c) download services, enabling copies of spatial data sets, or parts of such sets, to be downloaded and, where practicable, accessed directly; (d) What does Article 5 1. Member States shall ensure that metadata are created for the spatial data sets and services corresponding to the themes listed in Annexes I, II and III, and that those metadata are kept up to date. EuroGeographics Workshop Network Service, Paris, /06 where practicable mean?

14 Spatial data sets Spatial phenomena fall into two broad categories discrete and continuous. Discrete phenomena, like a building or a transportation network, are normally modelled as spatial objects with a distinct geometry (point, curve, surface, or solid) and a set of attributes. Continuous phenomena, like a temperature distribution, are normally modelled as a function that associates a value (or a tuple of values) to each point in space. A spatial data set contains a collection of representations of spatial phenomena.

15 Spatial objects A spatial dataset contains one or more spatial objects. A spatial object can represent either a discrete or a continous phenomena, i.e. it can be either a vector feature or a coverage feature. The spatial object types and associated attributes shall be compliant with the INSPIRE Generic Conceptual Model.

16 Query A query in the abstract sense, is a set of predicates expressed in a syntax collectively called a query and defined by a query expression. A predicate is an expression that can be validated to be true or false. A query is used to identify a subset of resources from a collection of resources whose property values satisfy a set of logically connected predicates. If the property values of a resource satisfy all the predicates in a query, then that resource is considered to be part of the resulting subset. In the context of INSPIRE, the spatial data set is a data set as defined by the Annexes I III. The result of applying a query will be a part of the data set, as required by the directive. Query

17 Query operations Function Description M/O General predicates Identifier predicate Spatial predicate Bounding box Temporal predicates Shall consist of: logical predicates: and, or and not comparison predicates: equal to, not equal to, less than, less than or equal to, greater than, greater than or equal to, like, is null and between Shall consist of predicate to check whether a resource identifier matches a specified value Shall only include bounding box Shall consist of temporal predicates: after, before, begins, begun by, contains, during, equals, overlaps, meets, overlapped by, met by and ended by M M M M

18 Pre defined dataset pre defined dataset pre defined part of dataset a full or partial MS extent of one of the INSPIRE themes. it will exist in a data store fixed CRS and encoding shall be described by metadata a pre defined part shall also exist in a data store it shall be described by its own metadata record

19 Direct access download service The directive states that MS shall operate download services enabling copies of spatial data sets, or parts of such sets, to be downloaded and, where practicable, accessed directly. The service metadata elements will define if the service is a direct access download service or not. If the download service offers a query capability, it is a direct access download service. If the service does not offer a query capability, then only a complete pre defined spatial data set or a pre defined part of a spatial data set can be downloaded.

20 where practicable The differentiation between direct access and non direct access has technical implications, but the technical aspects depends more on the capability of the MS. This implementing rule does not define any criteria for when direct access is practicable. This can be defined by the Implementing rule for the different themes, or by the MS themselves.

21 Download service functions Function Get Download Service Metadata Get Spatial Objects Describe Spatial Object Types Description Provides all necessary information about the service to a user (service provider, spatial objects available, access constraints ) and describes service capabilities to enable a client application to use the service (list of supported operations). The Get Spatial Objects operation allows spatial object instances to be retrieved. In the case of direct access, the retrieval can be based on an optional query defined by the Define Query operation. In the case of non direct access, the operation will retrieve a predefined data set or a pre defined part of a data set. The operation shall support user requested CRS belonging to the INSPIRE defined CRSs. The Describe Spatial object Type operation generates a description that defines zero or more of the spatial object types that the service offers. In the case of download service of a pre defined data set or predefined part of data set, the function shall return the description of the complete set of spatial object types contained in the data set or part of data set. In the case of a direct access download service, the function can have as parameter a set named spatial object types for which the description is requested. M/O /C M M C, M in case of direc t acces s

22 Download service functions Function Description M/O/C Define Query Link Download Service Defines a query to be used in the Get Spatial Objects operation. The query predicates, encoded using a query expression, can include spatial, temporal and non spatial constraints. The predicates shall express characteristics based upon the model of the data set as defined by an INSPIRE Implementing Rule on the interoperability of spatial data sets. The general characteristics are defined by the generic conceptual model. This function is applicable only in the case of direct access download service. The capability to define a query is mandatory, but a query can be omitted in a concrete Get Spatial Objects request. Allows the declaration of a Download Service for downloading of its resources through the Member State Download Service while maintaining the downloading capability at the Public Authority or the Third party location. C, M in case of direct access M

23 Pre defined dataset download Discover network discovery service Metadata catalogue Select selection process not specified Download EuroGeographics Workshop Network Service, Paris, /06 download service Archives Repositories

24 Direct access download Discover network discovery service Metadata catalogue Select Filter, query Download selection process not specified filter service EuroGeographics Workshop Network Service, Paris, /06 download service Archives Repositories

25 Simplified picture Member state data Harmonised MS data real or virtual Download Harmonised GML schemas File WCS? WFS and FE EuroGeographics Workshop Network Service, Paris, /06

26 CRS as defined by the Annex I theme European Terrestrial Reference System 1989 (ETRS89) for the areas within the geographical scope of ETRS89. The International Terrestrial Reference System (ITRS) or other geodetic coordinate reference systems compliant with ITRS shall be used in areas that are outside the geographical scope of ETRS89. The parameters of the GRS80 ellipsoid shall be used for the computation of latitude and longitude and for the computation of plane coordinates using a suitable map projection. For the vertical component, INSPIRE will mandate for the areas within the geographical scope of EVRS the use of the European Vertical Reference System (EVRS). Lambert Azimuthal Equal Area (ETRS LAEA) for spatial analysis and display; Lambert Conformal Conic (ETRS LCC) for conformal pan European mapping at scales smaller or equal to 1:500,000; Transverse Mercator (ETRS TMzn) for conformal pan European mapping at scales larger than 1:500,000. Download services whether for pre-defined datasets or direct access shall conform to these!

27 Other requirements Download Output Format The Download Services shall support any encodings defined by the corresponding specification of the INSPIRE themes. Multilingualism According to article 8c of the Directive, some elements of the Download service responses may be multilingual. The multilingualism applies only for exceptions raised by the INSPIRE service and for the abstract Get Download Service Metadata function. The default language used for these elements is left to each MS.

28 Performance The performance of an INSPIRE service represents the service response time, which must be kept for the given capacity. A service request is understood as a single call to a single operation of an INSPIRE service. Response time is the time measured on the server, in which the service operation returned the first byte of the result. Function Get Download Service Metadata Get Spatial Objects Description 3 seconds in normal situations *) 30 second initial response, then the service shall maintain a sustained response > 0,5 MB/s, alternatively 500 spatial objects/s in normal situations *) Describe Spatial Object Types 10 seconds initial response, then the service shall maintain a sustained response > 0,5 MB/s, alternatively the attribute values of 500 spatial object types per s in normal situations *) Define Query Performance criteria are not applicable. *) Normal situation represents periods out of peak load. It is set at 90% of the time.

29 Capacity and Availability Capacity The capacity of an INSPIRE service is given by a number of service request which are sent in a given time frame. Then the performance indicator has to be met for every individual service response. Value : 10 connections per second. Availability The availability of an INSPIRE service is the probability that the system is up and running. Value : 99%

30 Technical Guidance Normative references IETF RFC 2616 Hypertext Transfer Protocol HTTP/1.1 IETF RFC 2818 HTTP Over TLS ISO/DIS Geographic information Web feature service (211N2632 Text of ISO for DIS, Geographic information Web feature service) ISO/DIS Geographic information Filter encoding (211N2633 Text of ISO for DIS, Geographic information Filter encoding) ISO 19136:2007 Geographic information Geography Markup Language (GML) ISO 19119:2005, Geographic information Services ISO 19119:2005 PDAM 1, Geographic information Services ISO 19123:2005 Geographic information Schema for coverage geometry and functions OGC r3 OGC OWS OpenGIS Web Service Common Implementation Specification INSPIRE, INS ARC, Network Services Architecture Version 3.0 The following documents are candidate normative references if the functionality of the web coverage service is required for a given theme. OGC r5 OGC WCS OpenGIS Web Coverage Service (WCS) Implementation Standard OGC r5 Corrigendum 2 for the OGC Standard Web Coverage Service 1.1 (1.1.2)

31 Download services for predefined datasets or pre defined parts of dataset

32 Sequence diagram Inspire client Discovery service Metadata catalogue DB Download service Dataset repository request discovery search catalogue respond discovery respond search request download return dataset identify dataset return dataset

33 Function M/O/C Recommended implementation in case of download of pre defined datasets or pre defined parts of datasets Get Download Service Metadata Get Spatial Objects Describe Spatial Object Types Define Query Link Download Service M M C, M in case of direct access C, M in case of direct access M The Discovery service provides directly the metadata of the pre defined dataset or predefined part of dataset. Optionally the above metadata can contain a link to more extensive metadata. Note. In this simple case of download service, the service metadata and the dataset metadata will contain the same information. Issuing a GET command in HTTP protocol with the URL to the dataset as parameter initiates and completes the download. If applicable, the spatial dataset metadata shall contain description of the spatial object types instantiated in the dataset. Not applicable for this type of download services To be implemented by uploading the appropriate metadata to the INSPIRE network using PublishMetadata function of an INSPIRE compliant discovery service

34 Specific requirements The HTTP protocol is described in IETF RFC If a secure connection is required, then the HTTPS protocol should be used as described in IETF RFC The pre defined dataset or part or pre defined part of dataset shall be encoded in GML as described by ISO 19136, and the CRS shall be conforming to the implementing rule for Annex I Coordinate reference systems. The dataset can be transformed to another CRS by applying an INSPIRE conformant transformation service if required.

35 Direct access download service

36 Sequence diagram Inspire client Discovery service Metadata catalogue D B Download service Data repository D B request discovery search catalogue respond discovery respond search define query request download retrieve data return dataset return data

37 Conformance class name Operation or behaviour M/O Simple WFS Basic WFS Transactional WFS Locking WFS HTTP GET HTTP POST SOAP Inheritance Remote resolve Response paging Standard joins Spatial joins Temporal joins The server shall implement the following operations: GetCapabilties, DescribeFeatureType, ListStoredQueries, DescribeStoredQueries, GetFeature operation with the StoredQuery action only. One stored query, that fetches a feature by id, shall be available. Additionally the server shall conform to at least one of the HTTP GET, HTTP POST or SOAP conformance classes. The server shall implement the Simple WFS conformance class and shall also implement the Query action for the GetFeature operation and the GetPropertyValue operation. Servers that implement this conformance class shall also implement the Minimum Spatial Filter conformance class from ISO The server shall implement the Basic WFS conformance class and also implemented the Transaction operation. The server shall implement the Transactional WFS conformance class and shall implement at least one of the GetFeatureWithLock or LockFeature operations. The server shall implement the Key value pair encoding for the operations that the server offers. The server shall implement the XML encoding for the operations that the server implements. The server shall implement XML encoded requests and results within SOAP Envelopes. The server shall implement the schema element() function in Xpath expressions. The server shall implement the ablity to resolve remote resource references. The server shall implement the ability to page through the response features or values. The server shall implement join predicates using all Filter operators except the spatial and temporal operators. The server shall implement join predicates using spatial operators. The server shall implement join predicates using temporal operators. O O O O O O M O O O O O Conformance classes Feature versions The server shall implement the ability to navigate feature versions. M Manage stored queries The server shall implement the CreateStoredQuery and the DropStoredQuery operations. O

38 Function M/O/C Recommended implementation in WFS and FE Get Download Service Metadata M GetCapabilities operation of WFS GetFeature operation of WFS Get Spatial Objects Describe Spatial Object Types M C, M in case of direct access Any query shall be submitted to the service as a parameter to the GetFeatures operation (adhoc query), or, optionally, by a CreateStoredQuery operation and the query name passed as a parameter to the GetFeature operation. In both cases, the query shall conform to FE. DescribeFeatureType operation of WFS. Optionally an Adhocquery of WFS conforming to FE can be passed as a parameter to DescribeFeatureType in order to select a subset of feature types to be selected. Define Query C, M in case of direct access Adhocquery of WFS conforming to FE and passed as a parameter to GetFeature or DescribeFeatureType (not here), optionally as a CreateStoredQuery operation of WFS, with query expression conforming to FE. Link Download Service M To be implemented by uploading the appropriate metadata to the INSPIRE network using PublishMetadata function of an INSPIRE compliant discovery service. Additional operations of WFS that not required by the Implementing rule for download services, some will anyway be mandatory according to this Technical Guidance as they belong to a mandatory conformance class

39 Query implementation <<type>> Query <<type>> Projection <<type>> Filter <<type>> Sort <<type>> General <<type>> Spatial <<type>> Temporal <<type>> Identifier Function Description M/O General operators Shall consist of: logical predicates: and, or and not comparison predicates: equal to, not equal to, less than, less than M or equal to, greater than, greater than or equal to, like, is null and between Identifier filter Shall consist of predicate to check whether a resource identifier matches a specified value M Spatial filter Bounding box Complex spatial filter Temporal filter Shall only include BBOX (bounding box) Shall consist of spatial predicates: Equals, Disjoin, Touches, Within, Overlaps, Crosses, Intersects, Contains, Dwithin (a specified distance), Beyond (a specified distance), and BBOX Shall consist of temporal predicates: after, before, begins, begunby, Tcontains, during, Tequals, Toverlaps, Meets, OverlappedBy, MetBy, EndedBy, and Ends M O M

40 Conformance classes for query Conformance classes M/O/C to be supported in Direct download services 1 Query M 2 Ad hoc Query M 3 Standard Filter M 4 Minimum Spatial M Filter 5 Spatial Filter O 6 Temporal Filter M 7 Version navigation C, M if data specification support versions 8 Sorting O

41 Get Download Service Metadata implemented by the GetCapabilities operation of WFS This operation shall return all required metadata elements from OWS Common. In addition the capabilities response document shall contain the following sections: WSDL section (optional) This section allows a server to reference an optional WSDL document that describes the operations that the service offers (see Annex F). This section may be included in addition to the OperationsMetadata section for tools that know how to use the WSDL. FeatureType list section (mandatory) This section defines the list of feature types that are offered by a web feature service. Lightweight metadata is provided about each feature type as described in Table 11 Elements to describe feature types. Filter capabilities section (mandatory) The schema of the Filter Capabilities Section is defined in ISO and is used to advertise the expressions that may be used to form query predicates. Support for languages (mandatory) Language of the current capabilities response (see annex A of TG). Supported languages for the service operations (see annex A of TG).

42 Get Spatial Objects implemented by the GetFeature operation of WFS The GetFeature shall be implemented as described in ISO/DIS The request shall support both stored queries and ad hoc queries. The response shall support a clients request for CRS as long as the requested CRS is defined in the INSPIRE Annex I theme for coordinate reference systems. The GetFeature operation shall support the GetFeatureById stored query.

43 Describe Spatial Object Types implemented by the DescribeFeatureType operation of WFS The mandatory DescribeFeatureType operation returns a schema description of feature types offered by a WFS instance. The DescribeFeatureType operation shall be implemented by all web feature services. The operation shall support parameter describing which feature types to be described. If this parameter is not submitted, the complete application schemas supported by the service shall be described.

44 Link Download Service To be implemented by uploading the appropriate metadata to the INSPIRE network using PublishMetadata function of an INSPIRE compliant discovery service.

45 Additional operations of WFS These operations are not required by the Implementing rule for download services, but some referred here for completeness O O O O O O O GetPropertyValue GetFeatureWithLock LockFeature Transaction DropStoredQuery ListStoredQueries DescribeStoredQueries

46 To SOAP or not to SOAP Neither the draft Implementing Rule nor the draft Technical Guidance documents discuss the SOAP binding in a sufficient way. EuroGeographics Workshop Network Service, Paris, /06

47 Questions