Spatial information and maps in statistics

Transcription

1 Spatial information and maps in statistics THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION

2 Day 2 afternoon INSPIRE Directive Spatial Data Infrastructure Open Standards / Web Services Metadata

3 INSPIRE Directive

4 INSPIRE Directive European Directive: Infrastructure for Spatial Information in Europe European Union (EU) Spatial Data Infrastructure (SDI) sharing of environmental spatial information among public sector organisations to better facilitate public access to spatial information across Europe addresses 34 spatial data themes needed for environmental applications Annex I: basic themes (e.g. administrative boundaries) Annex II: supporting themes (e.g. ortho-imagery) Annex III: other themes (population, energy resources, habitats, industrial and agricultural facilities, )

5 INSPIRE Directive Data themes

6 INSPIRE Directive Architecture suggested by INSPIRE: Interoperable services (web services) To produce, publish, discover, access and use geographical information Via the internet For the whole of Europe On local, national and European level

7 INSPIRE Directive Based on a number of common principles: Data should be collected only once and kept where it can be maintained most effectively It should be possible to combine seamless spatial information from different sources across Europe and share it with many users and applications It should be possible for information collected at one level/scale to be shared with all levels/scales; detailed for thorough investigations, general for strategic purposes Geographic information needed for good governance at all levels should be readily and transparently available Easy to find what geo-information is available, how it can be used to meet a particular need, and under which conditions it can be acquired and used

8 INSPIRE Directive Adoption Preparation phase ( ) "co-decision procedure Preparation of implementing rules The INSPIRE Directive entered into force on the 15th May 2007 Transition phase ( ) The European directive is operational Continuation of preparation implementing rules Conversion to national/regional legislation Implementation phase ( ) Phased implementation of rules

9

10 INSPIRE Directive Implementing Rules Technical Organisational /Policy Accepted by EC and published in EU statutes (binding!) Supported by "Guidance document", with real technical specifications Available for: 1. Metadata 2. Data Specifications 3. Network Services 4. Data and Service Sharing 5. Monitoring & reporting 6. Spatial Data Services (new)

11 INSPIRE Directive - Components 1. Metadata Data about data information that describes geographical data and services To discover data and evaluate use 2. Data Specifications Interoperability: Consistent combination of geographical information and services from different sources Give access to geographical data via netwerk services Harmonization: Adapt and align definitions of spatial objects and models and the way they are encoded, defined in data specifications in specific domain or application area defined by data model

12 INSPIRE Directive - Components 3. Network services Discovery services Searching for geographical data and services based on content of their metadata, showing the metadata View services Visualize, navigate, zoom in/out, overlay of geographical data via maps, show legend, show accompanying metadata Download services Enable to download part or whole data sets or enable to question/query data set directly Transformation network services Transform data sets to reach interoperability e.g. coordinate transformation, data model transformation

13 INSPIRE Directive - Components 4. Data and Service sharing Access to geographical data and services under harmonized conditions Scope: By member states and their administrations By agencies of the EC To execute tasks for the common interest Where these tasks have an impact on the environment

14 INSPIRE Directive - Components 5. Monitoring and reporting Followup member states in their INSPIRE implementation and how it is converted into national legislation Monitoring of a set of indicators (measurable) Reporting on qualitative information about national SDI s 6. Spatial Data Services A more general class of services than the Network Services Free

15 INSPIRE Directive - GeoPortal EU community geoportal and the Member States shall provide access to their infrastructures through the geoportal as well as through any access points they themselves decide to operate

16 Day 2 afternoon INSPIRE Directive Spatial Data Infrastructure Open Standards / Web Services Metadata

17 SDI: Why? Geographical information is different from other information Multifunctional and domain crossing Example: Parcel (source: commune, Federal Public Service Finance, different application domains) Complex compared with other data Structure: geometry, topology Quality: accuracy, precision, scale dependency Big data and expensive Images, raster data Data collection requires huge effort and expertise Competences needed for maintenance of geodata Specialized software powerful hardware IT infrastructure is different from "standard" IT-infrastructure

18 SDI: why? Evolution of the SDI From: Closed" GIS for small application domain Software-own databases and format Difficult: only for GI-experts Data only for dedicated goals several data gathering Island GIS" not integrated in company processes and architecture To: "Open" and modular SDI Follows overall IT-trend Based on services and open standards Maintaining geodata at the source Using geodata in as many application domains as possible Offer geodata as geographical services

19 SDI: principles Basic principles of an SDI Gather once Gather once and maintain where most efficient (at the source) Seamless Combine data from different sources seamlessly Use for wide range of applications Share Gathered by one organization Used by many Instantly available and transparent Easy to discover and evaluate Terms of use are simple and transparent

20 SDI: principles Advantages Enhanced creation, maintenance and use of geographical data Organizations enhance their processes and efficiency Organizations get more value for their investments Users (intern/extern) receive better services Lot of new application possibilities

21 SDI: principles Example: real estate information (inlichtingenformulier vastgoedinformatie) Real estate/civilian/company wants to know status of property: Size, location, destination, permits, environment, Local governments gather information > takes more than 30 days! Problems: finding the access to actual information in different sources, no automation, not simple for functionary

22 SDI: components Infrastructure Durable, supporting environment Can be compared with road- or telecom infrastructure Consists out of components that interact (interoperability): Harmonized geodata Metadata Geographical services (web services) Policy for geodata Coordination and monitoring More info: SDI Cookbook:

23 SDI: components Harmonized geodata Agreed, multi-functional data models and specifications Data exchangable between organizations Data can be combined between organizations, domains, countries, regions

24 SDI: components Harmonized geodata Example : Hydrographical network Multi-functional: navigation, maintanance waterways, drinking water, inundation risks, recreation, infrastructures, Exchangeable : MOW, VMM, VMW, WatLab, Toerisme Vlaanderen, Waterwegen & Zeekanaal, de Scheepvaart, Combinable: spoiling or inundations in water basin: between countries (NL/BE), between regions (Flanders, Brussels, Wallonia), between government/private (industrial zones, insurance companies) See INSPIRE Hydrografie data specificaties

25 SDI: components Metadata Information that describes geographical data Inventarisation, discover and find, evaluation of use Standardized metadata model Everybody describes metadata based on the same set of elements Title, goal, data, contact, location, use restrictions, availability To use in Catalogue applications

26 SDI: components Geographical services (web services) Discovery services Viewing services Download services Transformation services Central access via GeoPortal > Services based on Open Standards!

27 SDI: components Policy for use/sharing geodata Transparent, clear and easy to apply rules to access and use geodata E-commerce Coordination and infrastructure Technical and organizational guidelines Monitoring and reporting Supporting environment

28 Day 2 afternoon INSPIRE Directive Spatial Data Infrastructure Open Standards / Web Services Metadata

29 Online mapping tools But how and where are these data coming from?

30 What are open standards? Standard maintained by not-for-profit organisation with open decision making procedure Standard is published and the document is offered freely Intellectual property of the standard is provided irrevocable and royalty-free No constraints with respect to reusability of the standard Source: European Interoperability Framework (EIF) published by EC IDABC Programme

31 What are open standards? In-company Standards Open Specifications Free Specifications Doc GIF PDF RTF Open Standards XML TCP/IP Source: Advies 86 Open Source Software Vlaamse Raad voor Wetenschapsbeleid

32 Open standards for geo information ISO TC 211 Geographic information/geomatics Open GeoSpatial Consortium (OGC) International organization >350 active members (government, private, academic) Mission : deliver spatial interface specifications that are openly available for global use Formal co-operation between ISO and OGC

33 Client Software Interoperability WWW WFS WMS WCS SOS CSW Data sources GeoTIFF/MrSID JP2000/ECW Oracle Spatial PostGIS PostGreSQL Data MIF/MID Shape/GML DB xml WMS: maps WFS: features WCS: coverages SOS:Observations CSW: catalog La directive INSPIRE en pratique GIM

34 Open standards for geo information Abstract specifications (19): Feature, ISO Metadata Implementation specifications (30): GML, WMS,... Specification Profiles: GML Simple Features Best Practices (20): GeoPDF Discussion papers (93): WCTS, WTS Deprecated documents (>100): Old Versions Requests for Information/comments Reference Model (2): Overview White papers

35 Open standards for geo information OGC Implementation Specifications Technical and detailed Interface between software components Simple Feature (4 specs) (SFS) Geography Markup Language (GML) Web Map Service (WMS) Web Feature Service (WFS) Web Coverage Service (WCS) Web Map Context Documents (WMC) Filter Encoding (FES) Styled Layer Descriptor (SLD) Symbology Encoding (SE) Catalog Service (CAT - CSW2.0) Coordinate Transformation Services (CT) Grid Coverage Services (GC) Location Services (OpenLS) Geographic Objects (GO) Web Serice Common (Common) GML in JPEG2000 (GMLJP2) GeoSpatial extensible Access Control Markup Language (GeoXACML) OGC netcdf encoding

36 Open standards for geo information WMS Web Map Service The OGC WMS specification is an interface specification, which allows the client software to access maps produced by the map services - in a standardized way. Different interactions: Q/A with respect to the offered map layers Question to offer map layer Simple requests It is possible to overlay different layers from different "Map servers" Map= visual presentation of underlying data

37 Open standards for geo information WFS Web Feature Service The OGC WFS specification is an interface specification, which allows the client software to access the actual vector data to download and edit - in a standardized way. Different interactions : Which datalayers are available, which functionality does the server offer (read, transactions, ) What is the content of the specific data layer Execute a spatial query Transaction: edit objects Vector data are mostly offered in GML

38 Open standards for geo information GML Geography Markup Language (GML) XML language for describing vector data For geographic (spatial) component and for attribute data GML describes the data, not the presentation of the data

39 Open standards for geo information CS-W Catalog Services for the Web The OGC CS-W specification is an interface specification, which allows the client software to access catalogues of geographical data and services - in a standardized way. Searching for data and services based on metadata Catalogues can be linked: "Harvesting" Remote access Catalogues can be accessed via a GeoPortal (client application)

40 When to use open standards? In organizations with a wide variety on geographical information platforms ArcGIS, MapInfo, AutoCAD, modelling software, ETL, Oracle, For applications that use geodata from different suppliers Real estate information, touristic applications Organizations that offer geodata to a wide variety of users and applications To build an SDI according to "Best Practices" To be conform with the INSPIRE directive, GDI-decreet, To build future, sustainable information systems

41 How to provide/set up Web Services? Example: GeoServer GeoServer is open source (and free) Provides geographic information in the form of interoperable services Popular software used by companies / governments as a basic building block for their SDI Advantages: Easy setup Intuitive Faster and more effective than its commercial counterparts OGC compliant Perfectly scalable Future ensured with the support of a stable and growing community of users and developers Supports WMS, WCS, WFS, More info:

42 Open Standards in practice: GIM GeoCMS Client application which connects to OGC WMS, WFS, WFS and WCS Services Easy configuration of counters Based on a Content Management System Export of GML, KML, Shape, GeoTIFF, PDF Symbology using SLD Configuration based on WMC Chart functionality Editing Tested using the most popular servers

43 Open Standards in practice: GIM GeoServices Offer maps and online Geo-Web Services Web Map Services - VHRSI - TomTom Streetmaps - Geocoding Reverse geocoding Address generalization Hit-based cost model (monitoring and control) Web application to monitor hits

44 Day 2 afternoon INSPIRE Directive Spatial Data Infrastructure Open Standards / Web Services Metadata

45 What are metadata Data about data - e.g.: data creation date what, who, where, when, how, why, Data describing a series of geodata, a set of series of geodata or geodata service Allow research, inventory, and facilitates the use of geodata Within the organization and between different organizations

46 Standardisation organisms Metadata OGC (Open Geospatial Consortium) ISO (International Organisation for Standards) (ISO/TC 211) CEN (European Committee for Standardization) (CEN/TC 287) DWIWG (Digital Geographic Information Working Group) NATO s Digital Geographic Information Exchange Standards (DIGEST) FGDC (Federal Geographic Data Committee) (FGDC-STD ) -National Spatial Data Infrastructure (NSDI) in USA New Zealand Government Geospatial metadata standard for New Zealand Government agencies DCMI (Dublin Core Metadata Initiative) (ISO 15836) Cross-domain information resource description: limited content but widely usable

47 Metadata - ISO Series ISO 19106: Profiles ISO 19109: Rules for application schema ISO 19110: Methodology for feature cataloguing ISO 19113: Quality principles ISO 19115: Metadata ISO 19118: Encoding ISO 19119: Service Metadata ISO 19133: Location-based services (Tracking and navigation) ISO 19139: Metadata -XML schema implementation GeoNetwork 2014_01 GIM

48 ISO ISO 19115: norm versus standard Some 400 mandatory, conditional or optional metadata sections, entities and elements Minimum set of metadata required to serve full range of metadata applications Method for extending metadata A community can use these to build its own standard (application profile) according to ISO GeoNetwork 2014_01 GIM

49 ISO Contents Introduction References Terms (not defined elsewhere in the ISO 191** series) UML models including codelists Core metadata list Data Dictionary Descriptions of extensions and profiles Abstract test suite Dataset Metadata application profile Extension methodology (informative only) Metadata implementation (informative only) Hierarchical levels of metadata (informative only) Implementation examples (informative only) Multilingual support for free text metadata elements (informative only)

50 UML Package Entity (UML Classes) Metadata entity set information MD_Metadata Identification information MD_Identification Constraint information MD_Constraints Data quality information DQ_DataQuality Maintenance information MD_MaintenanceInformation Spatial representation information MD_SpatialRepresentation Reference system information MD_ReferenceSystem Content information MD_ContentInformation Portrayal catalogue information MD_PortrayalCatalogueReference Distribution information MD_Distribution Metadata extension information MD_MetadataExtensionInformation Application schema information MD_ApplicationSchemaInformation Extent information EX_Extent Citation and responsible party CI_Citation information CI_ResponsibleParty GeoNetwork 2014_01 GIM

51 Example ISO 19115: MD_Identification

52 ISO Core Metadata for Geographic Datasets Mandatory and recommended optional elements Answers What, Where, When and Who questions

53 ISO Core Metadata for Geographic Datasets

54 GeoNetwork as Metadata Catalogue Web-based Catalogue software for metadata of geographic datasets, dataset series, Feature Catalogues, Services, applications, Metadata catalogue forms the glue in a Spatial Data Infrastructure Central access points that allows to Create Metadata and make it available Publish Discover Metadata Find Access Datasets, Services, described by these metadata Bind Facilitates the exchange of data between organizations and their audience Helps in reducing duplication of datasets Enhances information consistency and quality

55 Main Features End Users Instant search on local and distributed geospatial catalogues Search on multiple catalogues through one website Downloading of metadata data, documents, PDF's and any other content An interactive Web map viewer Metadata Editors Online editing of metadata with a powerful template system Support for different metadata standards Data upload Administrators Scheduled harvesting and synchronization of metadata between distributed catalogues Groups and users management Fine grained access control Active user and developer community

56 Searching metadata Default search Allows to perform a free text search, a search on the Resource Type and/or on the geographic location (A) Free text search Type a search term in Search Field. Terms will be auto-suggested after typing 3 letters Perform Search Fields that are discoverable can be configured (B) Resource types By clicking on the dropdown, a list with possible values will appear. You can choose one or more items from the list. The values that are supported are Series for geographic dataset series (collections) Service for geographic services Dataset for geographic datasets

57 Searching metadata Default search (C) Geographic search: A small overview map is shown on the screen that allows you to draw a Bounding Box Search options SortBy : Different ways to order the results Relevance (based on lucene score) Title Metadata changedate Rating Popularity Scale Descending Scale Ascending hitsperpage: Number of results per parge (default = 20)

58 Searching metadata Advanced search Works similar to the default search You can however be more specific in your search criteria as it offers different metadata elements to query upon. Search directly on the Keywords, organization or category (maps, applications, audio, video, ) Lists of possible values are determined on the basis of the contents of the Catalog By clicking on the arrows, lists of possible values will appear You can then select one or more items Remove previously selected entries by clicking on the X

59 Searching metadata Advanced search When section: Limits your search to a particular temporal extent Either Anytime Either by using the metadata change date Either by using the temporal extent Spatial representation type: (e.g. grid, vector, tin, ) Scale: use the slider to enter the min and max scale denominator Catalogue: select the local and/or remote /harvested catalogue instances

60 Searching metadata Output of search: List of metadata records that match your request For each record the result page shows a summary of the metadata consisting by default of the title, the abstract, the thumbnails and the keywords and a set of icons

61 Searching metadata Analyzing search results Different Result views are offered Title and keywords Title, thumbnail and keywords Title, abstract, thumbnail and keywords Results can be sorted using a variety of criteria

62 Searching metadata Analyzing search results Search results can be selected to allow operations (deletion, changing privileges, exporting, ) on multiple records at once (De)select one by one using checkbox Select all or all in the current page Possible actions on selected results Download & printing Delete Change Adding new metadata (Link to administration page)

63 Searching metadata Metadata description After completing your search, you can view details by clicking on the metadata title. Or by clicking view on the Action menu (appears on mouse over)

64 Searching metadata Metadata description Metadata descriptions can be viewed in different View modes : Simple, Default, By Group, By Package or XML Simple View Mode is organized in the following sections: Title and Abstract Information about the dataset Contact information Metadata contact Technical information

65 Searching metadata Metadata description The Default View mode is organized in 6 sections that are shown below each other and that can be opened and closed: Identification info Distribution info Reference system info Quality info Metadata Content Information

66 Searching metadata Metadata description The INSPIRE view is organized in 10 sections and addresses all the metadata elements relevant for INSPIRE Identification Classification of spatial data and services Keywords & INSPIRE themes Geographic location Temporal reference Quality and validity Conformity Constraints related to access and use Organizations responsible for the establishment, management, maintenance and distribution Metadata Needs to be enabled in the system configuration

67 Searching metadata Metadata description The By Group View is organized in 3 tabs: Iso Minimum has 2 sections - Data identification - Metadata Iso Core has 5 sections - Data identification - Lineage - Reference system info - Distribution info - Metadata Iso All has all sections for which metadata is present

68 Searching metadata Metadata description By Package View mode The Per Package is organized in 12 tabs One per ISO19115 package except for Extent and Citation and Responsible Party Information Packages The XML view shows the metadata in XML format

69 Searching metadata Actions on Metadata records View: shows the metadata Zoom To: zooms to data on the map Save as XML: shows the metadata as XML Export (PDF): shows the metadata as pdf Export (MEF) : saves the metadata in zip format and includes the thumbnails Additional actions for administrators/editors Edit: opens metadata record in edit mode Delete: removes metadata record Other actions: - Duplicate: copy metadata record - Create Child metadata record - Privileges: assign specific rights to specific groups - Status: change metadata status: draft, approved, submitted, retired, rejected,. 69

70 Editing metadata To produce a good metadata record, always try to gather as much details as possible on the resource that you want to describe Login as administrator or editor (or reviewer) Mechanisms for creating a new metadata record - Add new metadata by copying it from a template - Importing XML formatted metadata - Duplicate existing metadata - Bulk metadata import - Harvesting

71 Editing metadata Metadata templates A template is a metadata record that is prefilled with default values GeoNetwork comes with templates for several metadata profiles including ISO19115/ISO19119/ISO19110/. Difference between a normal metadata record and a template can be set in the metadata editor by selecting the record type Administrators can create new templates that are generally usable Editors can create templates for their groups Note: to discover templates select kind = template in the search menu as by default templates are not shown

72 Editing metadata Metadata editing Views Simple View: Simplified View that only shows metadata elements that are present in the template. Simple: user is not exposed to entire ISO Specification User can not add metadata elements that are not present in the template User can remove metadata elements and cannot add them back Advanced View: is based on metadata XSD User has access to all metadata elements and can always add them Complexity XML View INSPIRE View Custom Views: through development

73 Editing metadata Importing Metadata one XML file at the time Go to the administration page Click on the Metadata insert link Browse for a file or Copy/Paste Select File type Select options - - apply XSLT stylesheet to transform metadata - UUID management - Assign to a group Assign to a category - validate or not Click Insert

74 Editing metadata Duplicate existing metadata Search and select metadata Click on Actions / otheractions / duplicate menu In the form that appears select a group Click on create

75 Editing metadata Metadata validation After Editing Save -and check the newly created metadata. Checks verifies the conformance of the metadata record wrt - ISO19139 schema - ISO 19115/19115 schematrons - GeoNetwork recommendation schematron - INSPIRE schematron if installed - Custom schematrons to check additional constraints The validation report is given in the right column In case of validation errors correct them and run the check again

76 Day 2 afternoon INSPIRE Directive Spatial Data Infrastructure Open Standards / Web Services Metadata Questions? Thanks for your attention!