Daniela Cristiana Docan I 6th Sept. I INSPIRE Conference 2017, Strasbourg. EEA Data Quality Management supporting INSPIRE implementation

Size: px

Start display at page:

Download "Daniela Cristiana Docan I 6th Sept. I INSPIRE Conference 2017, Strasbourg. EEA Data Quality Management supporting INSPIRE implementation"

Eugene Ross
5 years ago
Views:

1 Daniela Cristiana Docan I 6th Sept. I INSPIRE Conference 2017, Strasbourg EEA Data Quality Management supporting INSPIRE implementation

2 Data Quality in INSPIRE INSPIRE Technical Guidelines use ISO Geographic Information-Data quality

3 Data Quality Elements mentioned in INSPIRE TG ISO Geographic Information Data Quality

Data Quality in INSPIRE ISO 19157 Geographic Information Data Quality INSPIRE TGs cover: 1.

4 Data Quality in INSPIRE ISO Geographic Information Data Quality INSPIRE TGs cover: 1.Data Quality elements/sub-elements 2. Data quality measures (tests to be applied on dataset) 3. Minimum data quality requirements /conformance quality level/

5 Data Quality in INSPIRE : Protected Sites TG -- Recommendations: DQ elements and sub-elements Corresponding DQ measures Minimum data quality requirements

6 Data Quality in INSPIRE : Protected Sites TG acceptance criteria or conformance quality level

7 Data Quality in INSPIRE : Protected Sites TG Item: fields, records, value, features, relationships, files in the dataset package

8 INSPIRE data quality requirements INSPIRE: Completeness/Omission INSPIRE: Rate of missing item INSPIRE: No recommendation/constrains

9 EEA s Data Flow stages Guidelines for Reporting Obligations, XML schema, Database schema, Quality control checks A priori DQ requirements - absolute positional accuracy-10m what we want Run automatic quality checks [QA scripts XQuery] Automatic QA report for MS ETL (Extract Transform Load) tools Automatic and manually quality checks Automatic and manually quality checks Conformance test (minimum data quality requirements) Metadata and/or standalone data quality report] A posteriori DQ results/values what we get Source: EEA Common Workspace/Generic QA/QC

10 EEA s Automatic quality report equivalent to measures in ISO standard Source: Eionet Central Data Repository (CDR

How will EEA s quality checks connect with ISO elements & measures? INSPIRE Requirements in INSPIRE Element: Completeness-Omission Standardised Measure: Rate of missing

11 How will EEA s quality checks connect with ISO elements & measures? INSPIRE Requirements in INSPIRE Element: Completeness-Omission Standardised Measure: Rate of missing items [Error rate] (e.g. real, percentage, ratio) Minimum data quality requirements: None EEA s CDF s quality checks Mandatory values User defined data quality measure: All records must have the SITE_CODE field filled Conformance test: acceptance criteria is [0%] errors in the dataset Source: Eionet Central Data repository (CDR)

12 Data Quality Rule Registry (DQRR) [catalogue of standardized and user defined data quality measures] = Elements in ISO = Measures in ISO 19157

13 Data Quality Rule Registry (DQRR)

Data quality checks different point of views 1. Minimal mapping unit [the smallest size of area allowed to be represented in a given data set] Topological consistency or conceptual consistency?

Hence a check is required to ensure that, when coordinates are reported, each coordinate is to 4 decimal places, adhering to the 10m accuracy required.

14 Data quality checks different point of views 1. Minimal mapping unit [the smallest size of area allowed to be represented in a given data set] Topological consistency or conceptual consistency? 2. C34 Coordinate accuracy These are required to be in format the ETRS89 (2D)-EPSG:4258 coordinate reference system, with a 10m accuracy. Hence a check is required to ensure that, when coordinates are reported, each coordinate is to 4 decimal places, adhering to the 10m accuracy required. CDF s guideline The number of decimal places for decimal degrees coordinates Precision or resolution Not absolute positional accuracy The error vector for a single point, (Source: Weir et al., 2001, p.413) Source: Logical consistency-conceptual consistency or Format consistency?

15 Conclusions INSPIRE Data Specification requirements/recommendations Data Specifications are not restrictive on the data quality (e.g. elements to be covered, measure (tests) to be applied, or minimum data quality requirements) Consistency in defining data quality elements and measures (tests) across different annexes or/and themes (e.g. Annex III - Topological consistency and Temporal consistency and validity)

Conclusions EEA s QA/QC workflow Fulfils the INSPIRE requirements on data quality New components of the data quality will be covered/improved (e.g.

data quality checks across EEA s production stage

16 Conclusions EEA s QA/QC workflow Fulfils the INSPIRE requirements on data quality New components of the data quality will be covered/improved (e.g. absolute positional accuracy) Data Quality Rule Register (DQRR) project promote the interoperability of the data quality by proposing: Common criteria to categorise data quality checks across EEA s production stages Harmonize DQ terminology across different Core Data Flows (e.g. record uniqueness, duplicate elements, duplicates entries, duplicate value, duplicities, and uniqueness of primary key) Assign/link the existing DQ checks to ISO quality elements and sub-elements Harmonise the Standalone Data Quality Reports

17 Q&A Thank you, Daniela Cristiana Docan

Draft version 13 th July Delivery manual for Article 12 data

Draft version 13 th July Delivery manual for Article 12 data Delivery manual for Article 12 data Draft version 13 th July 2018 Contents 1 Introduction... 3 2 Data preparation process... 3 2.1 Overview... 3 2.2 Data validation... 4 2.3 Tabular data standards for