Quality issues in LPIS: Towards Quality Assurance

Size: px

Start display at page:

Download "Quality issues in LPIS: Towards Quality Assurance"

Vanessa Flynn
5 years ago
Views:

LPIS Workshop 2008, 17-18 September, 2008 1 Quality issues in LPIS:

GeoCAP AGRICULTURE UNIT EC - Joint Research Centre Relevant Quality

1 LPIS Workshop 2008, September, Quality issues in LPIS: Towards Quality Assurance Valentina SAGRIS Geoinformation management GeoCAP AGRICULTURE UNIT EC - Joint Research Centre Relevant Quality Principles Quality Assessment (Inspection) Quality Reporting Quality Metadata

xxx LPIS Workshop 2008, 17-18 September, 2008

2 xxx LPIS Workshop 2008, September, Quality Policy Basic principles and elements

LPIS Workshop 2008, 17-18 September, 2008 3 Quality Policy - institution, procedure, data Quality policy is formalised ISO 9000-3 Quality

3 LPIS Workshop 2008, September, Quality Policy - institution, procedure, data Quality policy is formalised ISO Quality management and quality assurance standards Documentation is available for all personal Responsibility of the QP delegated to highest authority

LPIS Workshop 2008, 17-18 September, 2008 4 Quality Policy - institution, procedure, data Quality Assurance

4 LPIS Workshop 2008, September, Quality Policy - institution, procedure, data Quality Assurance procedures, data ISO Quality management and quality assurance standards Quality Inspection data Quality Control

Definitions LPIS Workshop 2008, 17-18 September, 2008 5 Two Strategies: Quality Assurance (QA) & A Tool: Quality Inspection (assessment) proactive approach QA philosophy is to build quality into the

5 Definitions LPIS Workshop 2008, September, Two Strategies: Quality Assurance (QA) & A Tool: Quality Inspection (assessment) proactive approach QA philosophy is to build quality into the system on a continuous basis, from conception through implementation and update is a recurrent activity compares deviations of identified attributes from pre-defined values and assesses the variation between the two Quality Control (QC) reactive or end user form of quality management, assess how well product matches expectation of customer

LPIS Workshop 2008, 17-18 September, 2008 6 Quality

6 LPIS Workshop 2008, September, Quality Policy - institution, procedure, data Quality Assurance procedures, data management procedures External Quality Audit Quality Inspection data data Quality Control

7 LPIS Workshop 2008, September, Quality Assurance (QA) consists of: 1. Management of requirements Data quality means how well data meets the requirements. The process of QA should be inspired by the desire to demonstrate compliance with the EU and MS Regulatory requirements Requirements should be documented and prioritised, the way to meat the (new) requirement should be explicitly described (data layer, table, procedure etc.) Each identified requirement should be addressed: Allocate recourses-nominate responsible-establish deadline

LPIS Workshop 2008, 17-18 September, 2008 8

8 LPIS Workshop 2008, September, Different ways to document requirements: (1) textual document

LPIS Workshop 2008, 17-18 September, 2008 9

9 LPIS Workshop 2008, September, Different ways to document requirements: (2) e-encyclopedia

LPIS Workshop 2008, 17-18 September, 2008 10

10 LPIS Workshop 2008, September, Different ways to document requirements: (3) Database

11 LPIS Workshop 2008, September, Managing of the requirements Example: change in requirement The Commission Regulation (EC) No 796/2004 Art 2 (1a) Agricultural parcel : shall mean a continuous area of land on which a single crop group is cultivated by a single farmer. However, where a separate declaration of the use of an area is required in the context of this Regulation that specific use shall further limit the agricultural parcel; 1. Changes in application process? -> New rules for farmer sketch 2. Changes in LPIS code data layer? -> New specification for operator; update cycle 3. Changes in operator application -> New tool for coding during digitalisation 4. New control procedure? -> New guidelines for inspector Allocate recourses - nominate responsible - establish deadline

12 LPIS Workshop 2008, September, Quality Assurance (QA) continued 2. Data governance - management of data quality consists of Establishing of the Conformance quality level Planning*of quality assessment (inspections) for the data Reporting data quality *Planning quality assessment for the data means establishing of quality assessment frequency and setting up reporting benchmarks for different types of data sets

LPIS Workshop 2008, 17-18 September, 2008 13

13 LPIS Workshop 2008, September, DATA Quality Assessment A tool for quality policy

14 LPIS Workshop 2008, September, ISO standards for GI DATA Quality: ISO Quality principals for geographic datasets ISO Quality evaluation procedures ISO Metadata ISO standard providing methodology for Data quality measures

15 LPIS Workshop 2008, September, Quality Inspection (assessment) a recurrent activity both the QC and QA strategies use quality test (assessment) consists of: a sequence of steps to produce quality evaluation procedures and report results quality evaluation report

16 LPIS Workshop 2008, September, Requirements and Specifications Data sets Conformance quality level Quality evaluation plan Fail Fail Quality Inspection: Quality procedures Quality Inspection: Quality report PASS

LPIS Workshop 2008, 17-18 September, 2008 17 Quality Inspection Step 1 For each dataset Identify Identify dataset dataset

Identify Identify measures measures Step 3 Step 4 Select Select scope scope Determine Determine results results Step 5 Determine

17 LPIS Workshop 2008, September, Quality Inspection Step 1 For each dataset Identify Identify dataset dataset specific specific elements elements and and sub-elements sub-elements Conformance Conformance quality quality level level Step 2 Identify Identify measures measures Step 3 Step 4 Select Select scope scope Determine Determine results results Step 5 Determine Determine dataset dataset conformance conformance (compare (compare results results with with requirements) requirements) Quality Report

18 LPIS Workshop 2008, September, Step 1. Identify Dataset Specific elements Data Quality Element completeness logical consistency positional accuracy thematic accuracy temporal accuracy Data Quality Sub-elements commission/ omission conceptual consistency codelist consistency format consistency topological consistency absolute or external accuracy relative or internal accuracy gridded data position accuracy classification correctness non-quantitative attribute correctness quantitative attribute accuracy accuracy of a time measurement temporal consistency temporal validity ISO 19113

19 LPIS Workshop 2008, September, Example: For data set: Reference parcels element: completeness sub-elements: commission/omission Step 2. Specify measure(s) number of excess polygons/number of missing polygons % of excess polygons / % of missing polygons in dataset Step 3. Apply scope full inspection sampling Step 4. Determine results all evidences of anomalies (excess/missing polygons) are flagged and counted; total inspection value or set of values is determined Step 5. Data set conformance inspection value or set of values from step 4 compared to conformance quality level - a threshold value(s) for data which has being specified by dataset specification or regulatory requirements

LPIS Workshop 2008, 17-18 September, 2008 20 Quality TEST For each dataset Step 4 Determine Determine results results Determine Determine dataset dataset conformance conformance (compare (compare

20 LPIS Workshop 2008, September, Quality TEST For each dataset Step 4 Determine Determine results results Determine Determine dataset dataset conformance conformance (compare (compare results results with with requirements) requirements) Step 5 Quality Report Report Report results results (quantitative) (quantitative) Report Report results results (pass/fail) (pass/fail) Aggregation Aggregation of of results results

21 Aggregation of the results LPIS Workshop 2008, September, combines quality results from data quality evaluations based on different data quality elements and sub-elements, obtained by quality test(s) and methods. There are several way of aggregations the meaning of which should be clearly understood before drawing conclusions 100% pass/fail AR = V 1 x V 2 x V 3, where V 1, V 2 and V 3 evaluation results Weighted pass/fail AR = w 1 V 1 x w 2 V 2 x w 3 V 3 Subset of result sufficient for product purpose 100% pass/fail aggregation for selected evaluation results, when applied selection of evaluation results shall be documented Maximum/minimum value Each data quality result is given value based on its significance (by default 1) and AR determined as max (or min) of scores

22 xxx LPIS Workshop 2008, September, Quality Metadata -as part of LPIS Metadata

23 LPIS Workshop 2008, September, ISO19114

24 Examples of data quality positional accuracy measures (ISO 19114) LPIS Workshop 2008, September,

25 LPIS Workshop 2008, September, Example: CR796/2004 art 6.2 _Scope: reference parcels subject to aid application _Element: thematic accuracy _SubElement: quantitative attribute accuracy _MeasureDesc: compare eligible land versus reference parcel area for the 75 percentile _EvalMethodType: 2 - external (direct)/ sampling _EvalMethodDesc: via statistical methods _ValueType: 4 - percentage _Results _Value: 95 % _ValueType 4-percentage _Date: _ConformanceLevel 90% (specified CR796/2004 art 6.2)

26 LPIS Workshop 2008, September, What are the advantages of QA framework over the current system will it facilitate quality inspections by Commission What are components, which are easily targeted and already in place? What needs to be done? Proposal for a path.

27 LPIS Workshop 2008, September, To DO List Implement Quality Assurance as prime strategy for MS Manage the requirements Formalise quality data governance (an agreement between EC and MS) Provide Quality Metadata records on regular basis as part of LPIS Metadata application

28 LPIS Workshop 2008, September, Thank you for your attention!

LPIS Workshop Applications and Quality

2009 MARS Conference, 18-20 th November, 2009, Taormina 1 LPIS Workshop Applications and Quality 6-8 th October, Tallinn, Estonia Wim Devos, Valentina Sagris & Pavel Milenov EC Joint Research Centre Institute