Using INSPIRE Services for Reporting and Exchange of Air Quality Information under CAFE Directive Test bed Results Alina Kmiecik, Dominik Kobus, Magdalena Bednarek, Piotr Krok, Anna Zamolska 26,6/2012, Istanbul
Presentation scope Lot general information Pilot project workflow elements: metadata data model model transformation Reporting prototype architecture INSPIRE service integration Conclusions
Project general information Title: Development and demonstration of technical IT solutions for data Exchange and reporting under CAFE Directive using INSPIRE services LOT 2: de-centralized with less than 100 stations Customer: The European Community represented by Commission of the European Communities represented by Institute for Environment and Sustainability of the Joint Research Center. Contract no. 384397 Project involved participants: PARTNER ROLE RESPONSISBILITIES INTERGRAPH Poland Chief Inspectorate for Environmental Protection (CIEP) Mazovian Voivodship Inspectorate for Environment Protection in Warsaw (VIEP) West-Pomeranian Voivodship Inspectorate for Environment Protection in Szczecin (VIEP) Contractor, Project leader, Technological partner national LMO regional LMO regional LMO Domain Analysis, Functional Design, Services Development Project Management, LOT2 Contact Point Data Provider Air Quality Data Expert Regional Data Provider Regional Data Provider Institute of Environmental Protection (IEP) Domain expert Air Quality Data Expert INTERGRAPH Italy Technological partner Project Management
Project partners: Polish LMO s National LMO: Chief Inspectorate for Environmental Protection Regional LMO s: West Pomeranian Voivodship Inspectorate for Environmental Protection Mazovian Voivodship Inspectorate for Environmental Protection Intergraph Project Lider Institute of Environmental Protection Warsaw, Poland
Project overall workflow D 1.1. Report Current implementation of reporting and data exchange D.1.2. Report Integration of INSPIRE services with the CAFE reporting Prototype assessments D. 2.4 Report Conformity of the reporting under CAFE Directive D. 2.3 Report Demonstration of the prototype for compliance to INSPIRE rules D. 2.1.2.1 Report Data specification for air quality data according to INSPIRE and CAFE requirements D. 2.1. Report Architecture of the reporting prototype D 2.2 Report Development of the prototype D 2.1.3.1 Report Development of download services D 2.1.2.3 Report Methodology of transformation D 2.1.2.2 Report ORIGINAL DATA REMODELED DATA METADATA D 2.1.1.1 Report D 2.1.1.1 Report Metadata profile Data processing
Reporting prototype - components INSPIRE-compliant download services established for air quality data reporting, GML application schemas defined for publication of air quality data, following the rules of INSPIRE data specification and publication, AQD database schemas for collecting and publishing of air quality data from CIEP, VIEP Mazovia and VIEP West Pomeranian Data Providers, FTPService for uploading of near-real time air quality monitoring into the publication databases view services for ear-real time and validated air quality data, metadata catalogue and discovery service for collecting and searching of AQ metadata, SEIS-CAFE Geoportal.
Data scope in the project CAFE zones and related metadata Monitoring data (near-real time) Monitoring data (validated) Monitoring facilities Assessment information information on delimitation of zones and related metadata in Member State. provisional near-real time air quality monitoring data on observed concentration of pollutants at measurement sites, in particular the exceedancesof information and alert thresholds. fully quality controlled monitoring data as reported under the Council Decision 97/101/EC on Exchange of Information. AQ metadata related to observation dataset(s) : information on monitoring networks and stations used for reporting. annual information on (non)compliance of air quality pollutant concentration to the limit / target values or assessment thresholds calculated per zones and stations, as reported under Decision 2004/461/EC.
Original resources INFORMATION SCOPE SOURCE AIR QUALITY DATA LEVEL DATA SOURCE CONTENT EXPORT FILE FORMAT CARD. RESP. INSTITUTION Entities national JPOAT Delimitation of zones and zone related information Provisional nearreal time air quality monitoring data Validated air quality monitoring data national regional Organizations responsible for air quality monitoring, data collecting and reporting in Poland excel 1 CIEP OR database Zones and Agglomerations used for air quality assessment and management in Poland excel 1 vector data Zones and Agglomerations in general SHP 1 vector data CS5 XR Zones and Agglomerations with a special focus to Ozone concentration Provisional Near Real Time air quality monitoring data for Mazovian voivodeship Provisional Near Real Time air quality monitoring data for West Pomeranian voivodeship SHP 1 txt html N N CIEP VIEP Warsaw VIEP Szczecin national JPOAT Validated air quality monitoring data for Poland MySQL 1 CIEP Monitoring metadata national JPOAT Air Quality Monitoring related information in Poland excel 1 CIEP Annual assessment information national OR database Air quality assessment information for Poland in a form of report or OR database excel 1 CIEP
Developing metadata final profile RESOURCE Zones and Agglomerations NRT for Mazovian voivodeship NRT for West Pomeranian voivodeship Validated air quality monitoring data for Poland Air Quality Monitoring Networks Air quality monitoring stations in West Pomeranian voivodeship Air quality monitoring stations in Mazovian voivodeship Air quality measurement configurations in Mazovian voivodeship Air quality measurement configurations in West Pomeranian voivodeship Air quality assessment information for Poland METADATA ORGANIZATION DATASET LEVEL Single metadata file describing central database of official delimitation of zones single metadata file describing Mazovian NRT database single metadata file describing West Pomeranian NRT database Single metadata file describing central database of fully validated data Single metadata file describing central registry of monitoring networks in Poland single metadata file describing the West Pomeranian Registry of monitoring stations single metadata file describing Mazovian Registry of monitoring stations single metadata file describing Mazovian Registry of measurement configurations single metadata file describing West Pomeranian Registry of measurement configurations Single metadata file describing central resource of official assessment information for EC none SERIES LEVEL Single metadata file describing the series of NRT datasets in Poland none none Single metadata file describing the series of regional registries of monitoring stations in Poland Single metadata file describing the series of regional registries of measurement configurations in Poland none Resource Title (mandatory) Resource Abstract (mandatory) Resource Type( mandatory) Resource Locator (mandatory) Unique Resource Identifier (mandatory) Coupled Resource (mandatory) Resource Language (mandatory) Topic Category (mandatory) Spatial data service type (optional) Spatial data service type: View (optional) Spatial data service type: Download (mandatory) Keyword value (mandatory) Originating controlled vocabulary (mandatory) Geographic description (optional) Geographic bounding box (mandatory) Temporal Extent (mandatory/optional - depending of the dataset) Date of publication (mandatory for datasets / optional otherwise) Date of last revision (optional) Date of creation (optional) Lineage (mandatory) Spatial resolution (mandatory) Specification (mandatory) Degree (mandatory) Conditions applying to access and use (mandatory) Limitations on public access (mandatory) Responsible party (mandatory) Metadata Point of Contact (mandatory) Metadata Date (mandatory) Metadata Language (mandatory)
Metadata discovery example
Data model specification for air quality data requirements CAFE requirements Draft CAFÉ Implementing Povisions (as of January 19, 2009) Requirements for scope of air quality information Technical requirements for air quality information exchange Requirements extracted from Airbase Data Dictionary INSPIRE requirements Recommended methodology for common model development Recommendations for INSPIRE data specifications Requirements of Generic Conceptual Model Requirements for Application schema encoding
Model elements Feature types 18 Data Types 10 Code lists 38 16/3/2010 SEIS - CAFE Project final workshop - Brussels 16.03.2010
Remodeling process - workflow DATA TRANSFER FILE FORMAT PHASE 1: PRELIMINARY PROCESSING OF PRODUCTION DATA STEP 1: Input data model analysis STEP 2: Output data model analysis DATABASE SCHEMA PRODUCTION DATABASE STEP 3: Model mapping STEP 4: Definition of data processing rules DATA TRANSFER FILE STEP 5: Execution of processing rules PUBLICATION DATABASE DOWNLOAD SERVICE PHASE 2 ON-THE-FLY TRANSFORMATION OF PUBLICATION DATA Apply XSLT SEIS - CAFE Project final Define workshop XSLT - transformation Brussels 16.03.2010 transformation DATABASE SCHEMA GML APPLICATION SCHEMA AIR QUALITY 15
Model Mapping - example ORIGINAL DATA MODEL SEIS-CAFE PILOT AIR QUALITY DATA MODEL FEATURE TYPE FEATURE ATTRIBUTE VALUE DOMAIN FEATURE TYPE FEATURE ATTRIBUTE VALUE DOMAIN Strefy nazwa strefy Datatype: Varchar2 Datatype: Zone zonename Minsize=1 CharacterString Maxsize=60 Strefy kod_strefy Datatype: Varchar2 Datatype: Zone zonecode Minsize=1 CharacterString Maxsize=60 Strefy zanieczyszczenia Datatype: Varchar2 Datatype: pollutant (foreign key to Zone Minsize=1 CharacterString Pollutant feature type) Maxsize=255 Strefy aglomeracja Datatype: Varchar2 Datatype: Zone zonetype Minsize=1 CharacterString Maxsize= 50 Strefy powierzchnia Datatype: Number Zone area Datatype: Number Strefy liczba_miesz Datatype: Varchar2 Datatype: population (foreign key to Zone Minsize=1 Number Population feature type) Maxsize= 255 Shp files Zone boundary source Datatype: (geospatial file/ LAU2 Zone geometry SDO_GEOMETRY list)
Original resources PROCESSED DATA Provisional near-real time data 336 instances of measurement data per day (CA:Observation instances) Validated monitoring data 1.194.148 instancess of measurement data (CA: Observation instances) Assessment 4.698 instances derived from original source (CA:AssessmentInformation instances)
Reporting workflow - now STATION LEVEL NETWORK LEVEL REGION LEVEL COUNTRY LEVEL MANUAL MEASUREMENTS AUTOMATIC STATIONS CS5 JPOAT-W JPOAT-K XR JPOAT-W R461 Questioannarie AUTOMATIC STATIONS MANUAL MEASUREMENTS
Reporting workflow oryginal source of information STATION LEVEL NETWORK LEVEL REGION LEVEL COUNTRY LEVEL MANUAL MEASUREMENTS AUTOMATIC STATIONS CS5 JPOAT-W JPOAT-K XR JPOAT-W R461 Questioannarie AUTOMATIC STATIONS MANUAL MEASUREMENTS
Reporting workflow INSPIRE services integration STATION LEVEL NETWORK LEVEL REGION LEVEL COUNTRY LEVEL AUTOMATIC STATIONS MANUAL MEASUREMENTS NRT_MAZ WFS Monitoring Facilities WFS CS5 JPOAT-W WFS JPOAT-K Validated Monitoring data XR JPOAT-W R461 Questioannarie AUTOMATIC STATIONS MANUAL MEASUREMENTS WFS NRT_MAZ NRT_MAZ WFS
Prototype architecture logical components Export validated monitoring data Production environment SEIS CAFE pilot publication environment Export assessment data Export monitoring facilities CIEP Production Environment WFS WMS INSPIRE download service Upload data Export NRT data txt files to ftp server VIEP Mazovia Production Environment INSPIRE view services CS-W INSPIRE Discovery services Get data Publish NRT data - website VIEP West- Pomeranian Production Environment
SEIS CAFE Pilot Geoportal WMS NRT BROKER WFS NRT BROKER CS-W WMS ZONES WMS MONITORING FACILITIES WFS MONITORING FACILITIES WFS VALIDATED DATA WFS ASSESSMENT On-the fly XSLT transformation WMS NRT MAZ WFS NRT MAZ Database VIEP MAZ WMS NRT WPOM Database VIEP WPOM WFS NRT WPOM Database Metadata catalogue Database CIEP FTP service File folder VIEP MAZ File folder VIEP WPOM FTP server File folder CIEP
Prototype Toward INSPIRE compliant services WFS Database schema (INSPIRE specific attributes included) support for voidable values, INSPIRE compliant domains (codelists and enumerations) PUBLICATION DATABASE CONFORMANCE TEST PASS FAIL Compliance to INSPIRE encoding rules --- x
Prototype Toward INSPIRE compliant services Adoption of database structures to GML encoding rules XSLT transformation WFS Database schema (INSPIRE specific attributes included) support for voidable values, INSPIRE compliant domains (codelists and enumerations) PUBLICATION DATABASE CONFORMANCE TEST PASS FAIL Compliance to INSPIRE encoding rules ---
Prototype Toward INSPIRE compliant services GetFeature response before XSLT transformation: GetFeature response after XSLT transformation:
Reporting prototype - services As a part of the reporting prototype following types of Inspire web services was developed: Direct Access Download services WFS obligatory required by project specification View services (WMS) supplementary services published for SEIS-CAFE Pilot demonstration purposes Discovery service (CS W) supplementary service published for demonstration of metadata files developed under the project.
Reporting prototype services NRT Data NRT Data are published by services: WFS_MAZ Mazovian Voivodship region WFS_VPOM West-Pomeranian region WFS_NRT_Data aggregated services for both regions Feature types published: Observation Exceedance Example of Get Feature request: http://sdiportal.intergraph.pl/wfs_nrt_wpom/wfservice.aspx?request=getfeature&service=wfs&version=1.1. 0&typeName=ca:Observation&maxFeatures=2
16/3/2010 SEIS - CAFE Project final workshop - Brussels 16.03.2010 29
Reporting prototype Download service - validated monitoring data WFS_ValidatedAQD Feature types published: Observation Pollutant Example of Get Feature request: http://sdiportal.intergraph.pl/wfs_validated/wfservice.aspx?request=getfeature&service=wfs&versio n=1.1.0&typename=ca:pollutant&maxfeatures=2
Reporting prototype Download service - monitoring facilities WFS_MonitoringFacilities Feature types published: MeasurementConfiguration MonitoringNetwork MonitoringStation Example of Get Feature request: http://sdiportal.intergraph.pl/wfs_facilities/wfservice.aspx?request=getfeature&service=wfs&version =1.1.0&typeName=ca:MonitoringStation&maxFeatures=1
Reporting prototype Download service - Assessment WFS_Assessment Feature types published: MeasurementConfiguration MonitoringNetwork MonitoringStation Zone Pollutant Exceedance AssessmentInformation AnnualStatistics OrganisationEntity Example of Get Feature request: http://sdiportal.intergraph.pl/wfs_assessment/wfservice.aspx?request=getfeature&service=wfs&ver sion=1.1.0&typename=ca:assessmentinformation&maxfeatures=1
Compliance to the SEIS-CAFE Pilot requirements ADVANTAGES: 1. Provisional monitoring data: collected by the two independent regional LMOs: VIEP Mazovia and VIEP West-Pomeranian published throughout the two separate WFS services WFS_NRT_MAZ and WFS_NRT_WPOM, can be streamlined by cascading of services. into the single entry point (WFS_NRT DATA service) at national level. The responsibility for published data remains in hands of collecting entity. 2. The provisional monitoring data collected within a monitoring network on 1 hour basis can be provided to the public in a near-real time mode using WFS services. 3. WFS services allow fast access to the most recent air quality information. DISADVANTAGES: 1. The responses of WFS service are provided in GML format. Even it is a human readable format, it is not a user friendly way for delivery of AQ data to the public. 2. Reporting of air quality data using download services requires change of existing procedures for reporting of air quality. 3. The number of monitoring data is a significant factor decreasing efficiency of WFS services. 4. Defining of OGC filters for obtaining the AQ data necessary for reporting is not a simple activity and requires deep expertise on Filter Encoding standards. 4. WFS services support searching and download of interesting data by using filtering options. 5. Use of WFS services for delivery of air quality data simplifies the process of reporting and eliminates the effort related to data processing and preparation of written reports.
Toward user friendly viewing of air quality data
Sample R461 questionnaire site http://sdiportal.intergraph.pl/cafe/r461.html 16/3/2010 SEIS - CAFE Project final workshop - Brussels 16.03.2010 38
Sample R461 questionnaire site http://sdiportal.intergraph.pl/cafe/r461.html
SEIS CAFE Geoportal presentation
26/06/2012 INSPIRE Conference - Istanbul 41
Thank you! Intergraph Corporation Alina Kmiecik IT Analyst Security, Government & Infrastructure Facilitator of INSPIRE TWG on Statistical Units & Population distribution-demography t: +48.(42).253.48.20 m: +48.(66).846.68.40 Intergraph Polska Sp. z. o.o. f: +48.(42).253.48.01 ul. Piotrkowska 275 alina.kmiecik@intergraph.com 90-361 Łódź, Polska 42