Development of guidelines for publishing statistical data as linked open data

Transcription

1 Development of guidelines for publishing statistical data as linked open data MERGING STATISTICS A ND GEOSPATIAL INFORMATION IN M E M B E R S TATE S POLAND Mirosław Migacz INSPIRE Conference 2016 Barcelona, 26 IX 16

2 Overall objective Support decision making processes involving provision of standardized, usable and open georeferenced statistical data.

3 What is linked open data? Internet collection of documents published online accessible at Web location identified by a URL, Documents mainly human readable and cannot be understood by machines. Linked open data data machine readable formats and connecting described using Uniform Resource Identifiers (URIs), thus enabling people and machines to collect the data, and put it together to do all kinds of things with it (permitted by the licence). source: (CC 2.0)

4 Linked open data URI for names RDF to describe data SPARQL to query for data source: (CC 2.0)

5 Uniform Resource Identifier (URI) to make a long story short : object described by an internet address A country, e.g. Belgium A dataset, e.g. Countries Named Authority List In official statistics it can look like this: gmina Węgorzyno source: (CC 2.0)

6 RDF &SPARQL Resource Description Framework (RDF ) is a syntax for representing data and resources in the Web RDF breaks every piece of information down in triples: Subject a resource, which may be identified with a URI. Predicate a URI identified reused specification of the relationship. Object a resource or literal to which the subject is related. is the capital of Belgia LUB is the capital of subject predicate object SPARQL is a standardised language for querying RDF data. source: (CC 2.0)

7 Five stars of linked open data Make your stuff available on the Web (whatever format) under an open license. Make it available as structured data (e.g., Excel instead of image scan of a table) Use non proprietary formats (e.g., CSV instead of Excel) Use URIs to denote things, so that people can point at your stuff Link your data to other data to provide context source: (CC 2.0)

8 Now powiat łobeski (LAU 1) lobeski

9 Aim powiat łobeski

10 Specific objectives identification of statistical units for which data can be published with harmonization of their geometries for respective years building standarized URIs for statistical units identification and analysis of potential data sources plan for transformation of existing data sources into open formats creation of RDF metadata for data sources feasibility analysis for publishing linked open data

11 Identification of data sources Three major databases: Local Data Bank biggest set of statistical information available for a wide range of years updated monthly Demography Database integrated data source for state and structure of population, vital statistics and migrations Development monitoring system STRATEG a system for facilitating and monitoring the development policy key measures to monitor execution of strategies at local, regional, transregional and EU level.

12 Identification of data sources Other data sources: publications tables communiques announcements articles

13 Identification of data sources Metadata: thematic category, format (PDF, DOC, XLS, CSV), spatial reference (country, NUTS, LAU, functional areas, urban areas), temporal reference (years) presence of identifiers (TERYT, NTS, NUTS) update cycle

14 Preliminary analysis of data sources Key aspects: openness redundance of information popularity (based on view and download statistics) Inclusion / exclusion of the data source

15 Statistical units harmonization Basis: NTS (Nomenclature of Territorial Units for Statistical Purposes) Name NTS NUTS/LAU Identifier Region 1 NUTS Voivodship 2 NUTS Subregion 3 NUTS Powiat 4 LAU Gmina 5 LAU

16 Statistical units harmonization Input data: administrative boundaries since 2002 for LAU 2 (gmina), excluding 2007 Harmonization process: structure standardization standardization of identifiers (creating NTS identifiers) aggregation to higher level units (LAU 1 > NUTS 1)

17 Statistical units harmonization Non standard statistical units: functional areas urban areas Groups of NTS units Derive mostly from strategic documents Changes of geometries in time to be determined

18 Statistical units URIs NTS as basic classification Name NTS NUTS/ LAU Identifier Region 1 NUTS /6 Voivodship 2 NUTS /6/22 Subregion 3 NUTS /6/22/40 URI Powiat 4 LAU /6/22/40/11 Gmina 5 LAU /6/22/40/11/01/1

19 Data transformation plan Test workflow from ontology to SPARQL endpoint Decide what will be published as Open Data three major databases other data sources Create ontology Map to existing databases Export to RDF data store Publish on linked data server Workflow tested on STRATEG database

20 Ontology methods and tools Ontop platform to query databases as Virtual RDF Graphs using SPARQL SPARQL 1.0 Support Supports interface for ontology development Intuitive/powerful mapping language Support for free and commercial DBMS SPARQL end point

21 Mapping ontology on database

22 SPARQL query on mapped data

23 SPARQL endpoint tools for the web Apache Jena Fuseki Fuseki is a SPARQL server. It allows REST style SPARQL Query. Ontop generated RDF s are imported to Apache Jena Pubby A Linked Data Frontend for SPARQL Endpoints Pubby makes it easy to turn a SPARQL endpoint into a Linked Data server. It is implemented as a Java web application. Provides data at given linked data uri

24 Fuseki SPARQL endpoint query

25 Query result facilitated by Pubby

26 Further works Consultation of the designed workflow during a study visit at the Madrid University of Technology Setting up an internal test linked data server to implement web tools Creating ontologies and workflows for databases and other data sources

27 Summary results so far Harmonized geometries for statistical units Identified data sources with comprehensive metadata Preliminary data transformation plan with tools tested

28 Poland s data opening strategy launched this year aimed at opening data resources of government institutions with respect to the 5 stars of linked open data goals the grant results (guidelines) in line with the strategy increased probability of acquiring financing for a fully fledged implementation

29 INSPIRE Thematic Clusters collaboration platform Statistical Cluster: statistical units population distribution (demography) human health and safety Informal meeting of Cluster members during the coffee break (15:30 16:00)

30 Questions?