Pilot Group IPR - Air Quality. Wet tests XML UTD and Zones Feedback from France

Transcription

1 Pilot Group IPR - Air Quality Wet tests XML UTD and Zones Feedback from France

2 Contents 1. French context 2. Wet tests 1. Zones Talend and WFS 2. Zones Talend and MySQL 3. UTD Talend and MySQL 4. Zones FME and PostGIS 3. Results 1. General difficuldes encountered 2. First conclusions 3. Proposals

3 1. The french context New system LCSQA (Central Laboratory of Air Quality Supervision) designed in 2011 to coordinate the nadonal plan on air quality Decision to build a new system Transi;on period Temporary system ParDal data coverage CollecDng assessment data from local organisadons Offering main informadon Designing new system New database modelling Building reference data repositories SelecDng tools

4 2. Wet tests Wet tests technical environment Using temporary database (MySQL) as source of data Temporary soludon DRUPAL GeoServer MySQL Database Need to idendfy a tool to transform data to GML DRUPAL / Geoserver Talend Open Studio FME Desktop

5 2. Wet tests Zones - Talend and WFS DRUPAL source of datas Geoserver as source of geospadal datas (zone muldpolygons) WFS used to produce geospadal data to GML format Talend to produce the GML feature CollecDon of zones with geometry Data request WFS Transforma)on ZONES GML format AQD format getfeature XML File generated

6 2. Wet tests Zones Talend and MySQL MySQL v5.0 source of datas Talend Open Studio used to transform data from MySQL database to GML/AQD format Difficul;es encountered Use of AQD.xsd Talend is not able to output GML Difficulty to deal with geospadal objects, even with Talend SDI (SpaDal Data Integrator) No ability to have several loop levels as needed by the complex structure of the AQD objects Transforma)on AQD format XML File generated ZONES

7 2. Wet tests UTD Talend and MySQL MySQL v5 source of datas Talend Open Studio used to transform data from MySQL database to GML/AQD format XML file with one stadon and one day of data Transforma)on AQD format XML File generated Up- to- date data

8 2. Wet tests UTD data repor;ng ques;ons Time Period: [ 00:00, 23:00 ] or [01:00, 24:00]? Or any period? Timestamp and period covered: 00:00 is for [00:00, 01:00]? 01:00 is for [00:00, 01:00]? Timezone for Dmestamp values according to the Guidance «repordng of Dme» and the documentadon of repordng schemata : YYYY- MM- DDTHH:mm:ss+01:00? YYYY- MM- DDTHH:mm:ss? Do we have to send 24 values a day? What happens if less than 24 values? Process: If values updated or verified do we have to send again? Within which period? What controls will be made from EEA data integradon? Data exhausdvity Data integrity (respect of the AQD format) Changes from last report (ex : new sampling point)

9 2. Wet tests Zones FME Desktop and PostGIS database PostgreSQL / PostGIS source of data Zones muldpolygons stored in PostGIS FME Desktop used to transform data and produce the feature CollecDon Transforma)on RESULTS : Full data Feature CollecDon with : o AQD_ReporDngUnit o list of AQD_Zone GML 3.2 compadble NaDve management of geospadal objects ExecuDon Time : ~ 10 s AQD format Transform to GML Feature Format to GML 3.2 Build Feature CollecDon XML File generated ZONES

10 3. Results General difficul;es encountered Use of AQD.xsd within the tools (especially with Talend as an output format) Understanding of the whole XML structure (GML + INSPIRE + AQD) Talend Open Studio limits with GML and loops SelecDng and using a tool needs Dme and investment with no guaranteed success ValidaDon of the XML files not possible when the AQD.xsd doesn t hold the reladonal structure

11 3. Results First conclusions TransformaDon tools Talend Open Studio v5.0 FME Desktop 2012 Advantages Drawbacks Open source SpaDal Data Integrator made for operadons on geospadal data Difficulty to use AQD.xsd as an output format No GML output format Limited to only two levels of loops SDI doesn t look maintained anymore GeospaDal data nadvely managed Reads GML 3.2 GML 2.0 compadble Enables to build full datasets, including AQD_ReporDngUnit ExisDng INSPIRE usecases and demos Not open source Doesn t nadvely write GML 3.2, needs specific transformadon

12 3. Results First conclusions Database MySQL supports spadal extensions for geographic features MySQL not compadble with GML PostGIS offers SQL funcdons to deal with GML data AQD elements AQD.xsd is not usable as an output format neither as an XML validadon file within tools AQD.xsd holds the definidon of AQD objects but doesn t hold the reladonal structure No available XSD to validate a specific report (ex : Dataset B Zones) Support documentadon is very complete The WFS and Geoserver scenario offers an alternadve to provide GML data when geospadal data is separated from business data. According to the architecture of exisdng systems, it can be an interesdng soludon.

13 3. Results Proposals AQD.xsd If AQD.xsd is only meant to be an object dicdonnary, we need complementary XSDs to validate each report structure If AQD.xsd is meant to be used to validate report files, it needs to hold the reladonal structure Reference files with a complete structure for each Dataset would help MS to understand further INSPIRE and AQD XML structure and target more closely For exisdng systems, a technical soludon could be to send geospadal data separately, in a shapefile for example, and report the reference to this file in the AQD xml file.