Technical implementation of INSPIRE: feed-back from experiences

Technical implementation of INSPIRE: feed-back from experiences Lisbon 27 February 2013 1 Modèle TN-02.018-1.2 Dominique.laurent@ign.fr

Plan Introduction Metadata Services Interoperability 2

Introduction Sources Implementation of INSPIRE in IGN France KEN INSPIRE workshops: KEN : Knowledge Exchange Network Members: National Mapping and Cadastral Agencies October 2012 : INSPIRE services February 2013: strategy to implement INSPIRE specifications on interoperability European projects (ESDIN, Humboldt) Other (INSPIRE conferences,..) 3

Introduction Colour coding Best practice : what is good, what works well Issue: what is not nice, what does not work Potential solution, lesson learnt from experience : what may help 4

5 Metadata

Benefits Main benefit of INSPIRE is to reveal hidden data sets 6

Which data sets for INSPIRE? Is it possible to have a broader scope than INSPIRE? We have made political decision to document all our products by INSPIRE metadata even if they are not in INSPIRE scope. The European Commission is happy to harvest also metadata for products out of INSPIRE scope (UK) We document all our products by INSPIRE metadata even if they are not in INSPIRE scope. but we make some filtering to avoid too many error reports from the European Commission (Germany) 7

Which data sets for INSPIRE? There is misunderstanding about the data sets to be documented by INSPIRE metadata: -All those in the scope of INSPIRE * -Only those that will be made interoperable We document all our data sets that are in the INSPIRE scope by INSPIRE metadata even if they won t/ can t be made interoperable but we make some filtering and we declare to the European Commission only those to be made interoperable (Sweden) 8 * With first option, there is risk to get bad indicators for Monitoring and Reporting

Which data sets for INSPIRE? Reference document We began the identification of INSPIRE data sets using the few lines describing a theme in the INSPIRE Directive. According to the -coming soon- specifications for annexes II and III, there will be more data sets and more data producers involved for some themes (Sweden theme Geology) The identification of INSPIRE data sets is an iterative process and in many countries, the list of INSPIRE data sets evolves every year. 9

Which data sets for INSPIRE? Granularity What is a data set? How big should it be? We have a lot of predefined data sets. INSPIRE requires to document them by metadata but it makes the search by users more difficult. -Too small: not convenient for discovery -Too big : not convenient for download The solution might be to manage parent-child relationship between the product (data set series) and the data set, in order to provide metadataat relevant granularity. 10

Metadata quality It is difficult to get good quality, INSPIRE compliant metadata from other data providers. Situation may be even worse for annexes II and III themes. Some countries are organising help for metadata providers: guidelines, Wiki, This help is generally provided in national language. 11

Technical aspects Which software are you using to host your metadata? 12

Methodology For a given data producer, how to identify the data sets to be documented by INSPIRE metadata? Are you the owner/producer of data? Ex: data from IGP is to be documented by IGP not by data users Ex: co-production => find agreement in order that only one producer does the work Is your data in the INSPIRE scope? In digital format Related to territory of a Member State From a public body Related to at least a theme of annexes I, II and III Can you benefit from exceptions? Public security, IP rights, privacy, Case of municipalities (less obligations / INSPIRE) 13

Methodology The matching with INSPIRE themes may be coarse (especially if same data appears in other data set) 14 IGN has declared the main theme (e.g. there are also buildings and administrative units in BD Parcellaire)

15 Network services

Discovery services Technical aspects GeoNetwork CSW is a good solution regarding INSPIRE requirements. (Germany UK) 16

Discovery services Technical aspects Synchronisation of data and metadata is a difficult topic. Storing data and metadata at same place helps to fulfill INSPIRErequirements. (Germany) Coupling data and metadata is also a solution. (France) 17

Discovery services Conformance The validation results may vary according the version of XML schema (ISO or OGC) and according to the parser. Even the Commission had 2 validators giving different results. The harvest by Commission generates too many reports with too many errors, making impossible to analyse them. (Sweden, UK) IOC TF might create an integrated XML schema (with the best of OGC and ISO) or OGC and ISO might converge. 18

View services Technical aspects What is the general architecture of your INSPIRE view service(s)? services or WMTS services, or both? 19 There is no obvious solution for view service architecture.

View services Technical aspects Which software do you use to provide the view services? WMS services or WMTS services, or both? 20 Balance between commercial and open-tools solutions.

View services Technical aspects you provide WMS services or WMTS services, or both? Do you provide WMS services or WMTS services, or both? WMS services or WMTS services, or both? 21

View services Layers The management of layers is complex: grouping of layers, layers for data out of INSPIRE, matching existing data to INSPIRE themes, Naming of layers is mandatory but impossible for existing data (France Sweden Belgium) 22

View services Tools There are open-source tools (Deegree, Geoserver, Mapserver) that are compliant with INSPIRE requirements (at least, with IR). Commercial solutions (ESRI, Intergraph) also claim INSPIRE compliance. (foss4gi survey) The INSPIRE extensions (e.g. multilingualism) raise some issues. Multilingualism is really required in some countries (Belgium) Multilingualism is also an issue for other services. 23

View services Technical aspects WMS is flexible but need more resources on server side and may be slow for large scale data. Many NMCAs use WMTS to get better performances. (Sweden, ) WMS may be used with reasonable performances if levels of zoom are well-defined. There is big interest in case of continuous update. (Rhine-Palatinat) 24

View services Conformance There are (at least) two tools to check compliance of view services to INSPIRE: neogeo and the GDI test suite. They do not check exactly same topics and look complementary. Moreover, JRC has also a tool for conformance : GeoPortal proxy. 25

View services Conformance Currently, we can not ensure 7 days / 24 hours availability of services. Services are always available in principle but in case of failure/issue, we can guarantee quick come back to normal situation only during office hours. (Sweden) 26

View services Use The display of INSPIRE layers gives bad rendering (poor INSPIRE legends); it is almost meaningless for some themes (AD, GN). Users prefer viewing cartographic products, traditional maps, at least as background. (Sweden) Display of vector data is more flexible and may adapt to varioususe cases but this requires additional efforts (let users combine layers, improve legends, ). (Germany) 27

Download services Technical aspects Do you provide download of predefined data sets or direct access through, for example, WFS? all : predefined data sets Denmark : WFS in future for harmonised data or WMTS services, or both? NMCAs currently deliver existing data (not yet harmonised) 28

Download services Technical aspects Will you be providing an ATOM feed to satisfy the Download Service obligations? 29

Download services Technical aspects What software (if any) do you use to provide download services? 30

Download services Technical aspects or WMTS services, or both? Download of tiled/ raster / coverage data raises issues. 31

Transformation services Use Transformation services are generally considered as useless by NMCAs: -Coordinate transformation: already offered by WFS -Schema transformation: requires too much knowledge from users Transformation services may be useful in case of very specific national CRS or to be offered as help to small data providers. 32

Network services Use Use of INSPIRE services is poorly monitored; When user feed-back exist, use of INSPIRE services is rather weak compared to use of other existing services. INSPIRE services might be more widely used once data is harmonised. 33

34 Data interoperability

Expected benefits Users Few users and use cases are expected at short term for INSPIRE data. We are not aware of any use case. We do it because of legal obligation BKG Some potential short-term users : -X-border users -Research projects -Environmental agencies (for reporting to EEA) Real benefit will come in long term with annex III data annex III data available and interoperable => more applications=> more use of annexes I and II data INSPIRE specifications are best practice and may be integrated /taken into account into data producers internal specifications. 35

Expected benefits Users INSPIRE data models are complex and we are unsure that GIS toolsmay handle INSPIRE data (IGN France). We are conducting a survey among GIS software providers: -to get better understanding of current status -to encourage GIS providers to take INSPIRE rules into account 36

Responsibility What to do when several data producers for same theme? Definition of responsibilities is at different stages in Member States Done In progress Not yet defined 37

Responsibility 3 responsibility models Single: one producer for a theme Node: several producers with coordination Multiple: whoever has, publishes Coordination may be done by various ways: Coordinator, leading actor Working groups Agreements between data producers 38

Responsibility Main benefit of INSPIRE interoperability is to force various data producers to sit together, to discuss and to find agreement. INSPIRE improves organisation of geo-information at national level. Technical issues may remain, when sharing responsibilities between data producers: -by feature types (Netherlands): no user friendly (no association, various LoD, several services) -by geometry / semantics (Norway) Definition of national standards (common data models) based on INSPIRE looks good solution. 39

Ambition What to do when a data producer has more data than required by INSPIRE? It is allowed to extend INSPIRE models Option 1 Do not extend INSPIRE model Keep existing products Have INSPIRE in complement of existing data 40 Option 2 Extend INSPIRE model Publish INSPIRE data in replacement of existing data

Ambition There is limited ambition for short term. INSPIRE will be in complement of existing products, very few extensions are expected. There will be interoperable INSPIRE data before the legal dead-line (Poland, Netherlands ) Keep it simple, apply only INSPIRE requirements (challenging enough). Expect some user feed-back before being more ambitious. Let s INSPIRE prove itself 41

Priorities Priorities are generally not driven by expected benefits / user requirements. Priorities are mainly driven by: -INSPIRE road map (annex I and/or new products) -Feasibility (AU, CP, GN, AD, PS before HY, TN) 42

Methodology Step 1 : Identification of the source data sets The list of data sets already declared for INSPIRE (metadata) may have to be reviewed for interoperability Distinction between: Production database External products Case of similar data sets Reference data set => to be made interoperable Copies 43

Methodology Step 2 : matching between source data sets and INSPIRE model 44

Methodology Step 2 : matching between source data sets and INSPIRE model 45 Matching tables Derive automatically matching table from INSPIRE data model Avoid fastidious manual work Avoid interpretation errors No perfect model of matching table High level (features, attributes) : missing correspondences Detailed (data types, code lists) : difficult to handle Often an iterative process Matching table more detailed Possibly new data sources

Methodology INSPIRE specifications are complex and require a lot of knowledge. Involve several persons to define the matching between existing data and INSPIRE -Knowledge of source data -Knowledge of INSPIRE theme specification -Knowledge of transformation process, transformation tool. 46

Step 3: analyse, decide Missing data Methodology Generally not an issue as most concepts in INSPIRE are voidable (to be provided if available) But a recommendation for new data capture Additional data Keep it out of INSPIRE Integrated it in INSPIRE => extend the INSPIRE model 47 Common data with INSPIRE The way to make it interoperable is let to data producer Data transformation => short-term term solution Change internal model => long-term solution

Step 3: analyse, decide Methodology How to deliver INSPIRE data CRS : INSPIRE gives the choice between several CRS Geocentric coordinates Geographic coordinates Projected coordinates Format : INSPIRE formats GML for vector data GML coverage, TIF, JPEG 2000 for gridded data Other (shapefile?) Media Direct access Predefined data sets 48

Methodology Step 4: choose or define the target schema The target schema may be : The INSPIRE schema (AU, GN, CP, AD, ) One of the INSPIRE schemas Annexes II III themes offer some flexibility Different application schemas Different aspects (physical waters/ water network for HY or vector/grid/tin for EL or 2D/3D for BU) Core / extended models (BU, LC,.) Generic / example models (natural risks / flood) Extensible code lists or code lists only recommended Producer-defined defined extensions 49

class ProviderDefined BuildingsBase «featuretype» BuildingsBase::AbstractConstruction + inspireid: Identifier «voidable, lifecycleinfo» + beginlifespanversion: DateTime + endlifespanversion: DateTime [0..1] «voidable» + conditionofconstruction: ConditionOfConstructionValue + dateofconstruction: DateOfEvent [0..1] + dateofdemolition: DateOfEvent [0..1] + dateofrenovation: DateOfEvent [0..1] + elevation: Elevation [0..*] + externalreference: ExternalReference [0..*] + heightaboveground: HeightAboveGround [0..*] + name: GeographicalName [0..*] «featuretype» BuildingsBase::AbstractBuilding «voidable» + buildingnature: BuildingNatureValue [0..*] + currentuse: CurrentUse [0..*] + numberofbuildingunits: Integer [0..1] + numberofdwellings: Integer [0..1] + numberoffloorsaboveground: Integer [0..1] «featuretype» BuildingsBase:: Building «featuretype» Buildings2D::Building + geometry2d: BuildingGeometry2D +parts «voidable» Buildings2D 0..* «featuretype» BuildingsBase:: BuildingPart «featuretype» Buildings2D::BuildingPart + geometry2d: BuildingGeometry2D [1..*] Principles : Do not remove any INSPIRE requirement Do not overwrite any INSPIRE requirement Recommended method: -Import INSPIRE application schema -To add new properties, specialize INSPIRE feature types -New feature types may also be added 50 Building + address: AddressRepresentation [0..*] + officialarea: OfficialArea [0..*] Buildings Provider Defined Extension BuildingPart + address: AddressRepresentation [0..*] + officialarea: OfficialArea [0..*] Extension by data provider

Methodology Step 4: choose the method and the tools for making data compliant with INSPIRE In case of data transformation, 2 main steps Coordinate transformation Schema transformation 51

Schema transformation Possible tools Commercial tools FME possibly with Arc GIS pour INSPIRE Go Publisher (Snowflakes). open-source tools HALE (Humboldt Alignment Editor) http:// community.esdi-humboldt.eu Talend Studio http:// fr.talend.com/products-data data-integration/talend-open-studio.php exows devloped par le BRGM http://sourceforge.net/projects/exows/ GeoConverter devloped Navarra region (Spain) http://www.geobide.es/productos/index.aspx. 52 Home-made made tools (if developer skills)

Schema transformation (at least) 2 main methods: option 1 Make first model changes in a relational database and then move to GML The last phase may be ensured by WFS servers (Degree, GeoServer, XtraServer, ) Examples Arc GIS for INSPIRE Logical structure of annex I themes implemented by ESRI in relational GeoDataBase data producer makes its transformations of data model (e.g. avec c FME)=> relational INSPIRE model the tool makes the format transformations and ensure the WFS flux Implementation in IGN 53

Schema transformation INSPIRE schema (GML) IGN data - Shapefile Base de données structure IGN Scripts SQL Base de données structure INSPIRE PostGre BD relationnelle GeoServer User query Data in GML and in INSPIRE schéma Out-of-line On-line) 54 * Similar method was used by partners of ESDIN project to offer data through WFS, using degree instead of GeoServer

Schema transformation option 2 : make first the format transformation and then define the transformation rules HALE : Humboldt Alignment Editor source and target schemas: GML range of available transformations (rename, select, fre- classify, give default values, Transformations to be defined once CST : Conceptual Schema Transformer makes the predefined transformations on source data To be launched on each data set (e.g. each update) 55 * Tool from research project

Schema transformation Source schema (GML) Target Schema (GML) source data (GML) in source schema Humbolt Alignment Editor Transformation rules (OML) Conceptual Schema Transformer Transformed data (GML) in target schema 56 * HALE may now work with other input formats

Methodology Step 5: make the required transformations Step 6 : check compliance to INSPIRE requirements Annex A of Technical Guidelines (Abstract Test Suite) gives the steps (theory) The GML schema (.XSD file) may be used to check conformance to data model Other potential tools Schematron : for constraints RadiusStudio (1spatial) : for rules 57? Likely not everything can be checked automatically Political issue : who should check

Thanks for your attention Any question? 58