Introduction : Challenges of ESPON DB. CHALLENGE = set of interrelated

Transcription

1 Challenges for the ESPON Database Development e e Project ESPON DB 2013 (Priority 3) Espon Seminar, Bordeaux, December 2008

2 Introduction : Challenges of ESPON DB CHALLENGE = set of interrelated tasks toward a specific objective.

3 Introduction : Challenges of ESPON DB 9 challenges has already been defined for the first period we can imagine to reach the figure of 12 in the future! Metopes of the Temple of Zeus at Olympia

4 Challenge 1 : Collection of basic regional data Objectives : Collection of basic indicators in new NUTS 2006 territorial division derived from EUROSTAT & EEA Situation : Complete data not available on Eurostat website in Sept Problem with CLC C conversion to NUTS 2006 Strategy : Meeting with Eurostat (Nov. 2008). Delivery of provisional data in old NUTS 2003 territorial division. Coord : RIATE

5 1.1 What are the data available on Eurostat currently? Example: GDP in 2005 (Euros) Situation of data during the ESPON seminar NUTS0 division NUTS1 division NUTS2 division NUTS2-3 division NUTS3 division Missing values: - NUTS0 : Liechtenstein - NUTS1 : Iceland, Norway, Switzerland - NUTS2 : Iceland, Norway, Switerzerland, United Kingdom - NUTS2-3: Iceland, Norway, Switzerland, United Kingdom, Poland, Italy, Spain - NUTS3: Iceland, Norway, Switerland, United Kingdom, Poland, Italy, Spain, Germany.

6 1.2 Short term solutions Deliver the statistical information in NUTS 2003 version, the must complete for data deliverable currently. For missing values, compute with different official sources (National statistical offices). Use scientific reports or study for complete the data set (previous ESPON DB, Parliament projects, DG AGRI database...)

7 1.3 Deliverable for ESPON projects Complete data sets for some basic indicator (GDP, population) in NUTS 2003 version at each scale A precise metadata file showing the different sources used to compute data A synthetic table summarizing the situation for some basic indicator.

8 Challenge 2 : Harmonisation of time series Objectives : Harmonisation of time series for the period Situation : non compatibility of NUTS1995, NUTS1999, NUTS2003, and NUTS2006 Strategy : Collection of data at each period «as they are» and elaboration of harmonisation procedures. Coord : IGEAT

9 21 d l 2.1 Expected results ( ex. Territorial dynamics in Europe: Trends in population development, ESPON, 2008)

10 2.2 The problem of changing territorial divisions In all countries of the world, regional divisions are changing from time to time

11 2.3 Formalisation of regional changes (Ben Rebah, 2008)

12 2.4 Empirical solutions (ex. Danish regions) Solution 1: collect data at a lower territorial level (usually impossible) Solution 2: make sure that keeping the nuts3 level won t affect the results, because of the specific studied d domain Lines in red show the nuts3 in 2003, lines in green show the nuts3 in 2006 Solution 3: find a workaround by using proxy indicators Solution 4: work at a higher territorial level

13 Challenge 3 : World / Regional data Objectives : combine datasets at world level (by states) and datasets at European level (by NUTS regions) Situation : no direct compatibility between world data by states (UN) and EU data by Region (e.g. Eurostat) Strategy : Elaboration of tools for compatibility between NUTS and WUTS data (key question of state : NUTS0=WUTS5?) Coord : RIATE & UNEP

14 3.1 Description of the challenge World data base Source : United Nations Basic units : States Agregation :WUTS Harmonisation? Regional data base Source : Eurostat Basic units : Regions Agregation :NUTS

15 The World Unified Territorial i ls System (WUTS)

16 Mapkit tool lfor World and Neighbourhood h

17 3.4 The key problem of state level States are the basic level (WUTS5) of the world data base and States are the upper level (NUTS0) of the regional data base But the data related to state level are generally different according to World data provider (UN) and to regional data providers (Eurostat) because of : Different definitions of indicators (e.g. Unemployment ) Different e definitions of countries (e.g. remote territories to es France not member of EU) Different methods of measure (GDP in $ instead of, alternative measures of PPP, ) Different approach of prevision (Demographic scenarios of UNPP and Eurostat).

18 3.5 Ex. What is the population of ESPON34 the 1st Jan. 2002? Good news! The global result is the same according to UN and Eurostat : 572 Millions (±0.1%) Bad News! Many results are strongly different at country level Cyprus, France, Romania, Estonia, Turkey,

19 Challenge 4 : Regional / Local data Objectives : Develop case studies providing zoom on specific territories at local level (rural areas, cross border areas, intra-urban differentiation, ). Situation : no complete data +geometry available at LAU1 or LAU2 level. Strategy : Elaboration of coherent local l sources (data + geometry) with Eurostat, Eurogeographics and national statistical institutes Coord : TIGRIS

20 4.1 Expected results (1) : zoom on infra-regional i trends

21 4.1 Expected results (2) : zoom on cross-border territories i

22 4.2 Evaluation of the complexity of the task WAITING FOR EUROSTAT? or BUILDING STATE BY STATE?

23 Test on Romania Problems: - Changes of geometries - Changes of codes and names - Difficulty to link data and geometry

24 Test on Romania Solutions: -Definition of a reference year (2000) where data and geometry are available - Reconstruction ti of past data using lower level (villages) or estimations - Use of official administrative document for history of units.

25 4.44 Other sources (1) : Topographic data (ex. EuroGlobalMap) lm

26 4.44 Other sources (2) : Grid data (ex. EEA/JRC)

27 Challenge 5 : Social / Environmental data Objectives : Making easier the combination of socio-economic data (EUROSTAT) and environmental data (EEA) Situation : Initiatives for grid harmonisation. Development of GRID<-> NUTS tools Strategy : Implementation of data exchange tools in ESPON DB. Networking with Eurostat/EEA/JRC on this topic (in the framework of INSPIRE). Coord : UAB (ETC-LUSI)

28 5.1. Description of Challenge 5 Combining socio-economic data measured for administrative zoning (Nuts level) and environmental data defined on a regular grid (Corine Land Cover) Corine Land Cover Data Gross Domestic Product (GDP)

29 5.2. Review of published methodologies POPULATION Downscaling population density in the European Union with a land cover map and a point survey. Francisco Javier Gallego, JRC. GDP G-Econ Research project developed by the University of Yale on Gross Cell Product (GCP): New Metrics for Environmental Economics: Gridded Economic Data by William D.Nordhaus.. DIFFERENT METHODS FOR DIFFERENT VARIABLES!!!

30 Common approach Integration of data based on the 1km reference grid Integration Methods: Proportional calculations Maximum area criteria

31 Preliminary i results Gross Domestic Product in Purchasing Power Selection of NUTS 3_ID: LU and BE (Value of GDP is given by NUTS 3) 2.- Intersect with 1 km Reference Grid 3. - Calculation of area percentage for each cell by NUTS GDP formula to be applied in each cell: Σ GDPnut * % Area cell = GDP cell

32

33

34

35 Conclusions Integration of socio-economic and environmental data by 1 km grid is a good solution, since data can be harmonised without losing resolution. Depending on the nature of socio-economic variables, a different integration approach should be undertaken. The method can be applied to both vector and raster data. This approach facilitates the integration into the EEA assesment LEAC system.

36 Challenge 6 : Urban data Objectives : Insure compatibility between the different definitions of cities and urban areas currently available. Situation : Great diversity of definitions and difficulty to combine data from different sources. Strategy : storage of metadata and conceptual clarification. Dictionnary of cities with common codification. Coord : Géographie-cités

37 Cities as complex geographical object Sub-City Urban Objects DataBase Source Sub City-District Urban Audit DataBase Core Cities Political entities Source Urban Audit Morphological agglomerations Databases Source Fonctional urban areas MUAs Espon Databases Source LUZ Urban Audit UMZ EEA LUZ Proxy Nuts3 Urban Audit FUAs Espon & revised 1.4.3

38 6.2 Variation of definition iti (example of Bordeaux) Spatial entities Databases Sub-City Districts SCD (Urban Audit) no spatial delineations available yet Political entities Core Cities (Urban Audit) Morphological agglomerations - MUAs (Espon 1.4.3) - UMZ (EEA) Fonctional urban areas - LUZ (Urban Audit) - LUZ Proxy Nuts3 (Urban Audit) - FUAs (Espon & revised 1.4.3) no full spatial delineations available yet

39 6.3 Benchmarking of UMZ and MUA (example of Saarbrücken) MUA : Saarbrucken is well identified (Pop = inhab. / Surf = 620 km²) UMZ : Which polygon(s) should we use? Different solutions!

40 6.4 Collection and harmonisation of metadata t (W.I.P.)

41 Geographical coverage of databases

42 Challenge 7 : Extra-ESPON data exchange Objectives : - Exchange of data and expertise with main data providers - Identify long-run synergies - Promotion of ESPON DB Situation: - Ongoing contacts with Eurostat (update NUTS2&3) and EEA (grids) Strategy : - Set up of regular meetings, strategic and technical, with institutions and CU -1 DB team <-> 1 external institution Coord : University of Luxembourg

43 7.1. Main institutions and flows Developing 2-way FLOWS of European Commission i DG REGIO; AGRI; ENV Others (local): National Statistics Offices, Regional observatories, etc DATA NUTS WUTS GRIDS EXPERTISE & KNOWLEDGE in Spatial Data Analysis, Integration, Management (meta s) Publication (web) ESPON DB2013

44 7.2. Example: EUROSTAT REGIO and GISCO services DATA Harmonized and updated official EU data (statistics and spatial unit definitions) at all scales EXPERTISE Management of different scales (LAU to NUTS) and geographical objects (Urban Audit) Metadata (INSPIRE) Experience with national Stat. t bodies (Beyond EU (MEDSTAT, candidate countries, )) Web publication, etc.. LONG-RUN SYNERGIES Identify data flow procedures at different scales for regular updating between ESPON DB and Eurostat (statistics, geometries) Sharing expertise - both way! - in data management (aggregation/disaggregation/combination of objects) and spatial analysis (including typologies, etc..)

45 7.3. Examples of possible win-win i strategies EUROSTAT > ESPON DB: geometries and statistics update, inspire metadata example ESPON DB > EUROSTAT: generalisation methods, interpolation methods and recasting of (backed-up) past data, EEA <-> ESPON DB: see challenge 5 (grids, UAB), inspire metadata example DG REGIO (GIS) <-> ESPON DB Comparing aggregation/disaggregation methods and results OECD <-> ESPON DB Fitting WUTS and NUTS, techniques for synthetic indicators

46 Challenge 8 : Intra-ESPON data exchange Objectives : 1) Provide an efficient interface between ESPON projects (Priority 1 and Priority 2) and statistical or cartographic institutions (Eurostat, EEA, Eurogeographics ) 2) Stimulate the launching of future ESPON Projects by incorporating indicators from neighbouring countries (Balkans, Ukraine, Belarus, Moldova, Maghreb, Turkey ) or elaborating long term ( ) series for a selected number of indicators Situation: One contact person for each other ESPON Projects Strategy: A data circulation model that 1) Answer quickly to urgent requests from running projects 2) Help ESPON to elaborate long term workplans Coord : RIATE & LIG

47 Data Flow in the ESPON 2013 DataBase: First Version (September to November 2008) Supported by the ESPON 2013 DataBase Version 1.0 delivered at the end of November 2008 External Data Providers Normalization with the import tools RIATE LIG Normalized Data Files ESPON Projects ESPON Projects INPUT DATA Raw Data Files Validated Data Files Insertion with the import tools OUTPUT DATA External World Primary manual data validation ESPON DB V 1.0

48 Data Flow in the ESPON 2013 DataBase: Second Version Supported by the ESPON 2013 DB Version 2.0 to be delivered at the end of February 2009 A clear distinction between the Internal ESPON DataBase and the External ESPON DataBase External Data Providers ESPON Projects INPUT DATA Automatic data checking and normalization Raw Normalized Data Files Data Files LIG RIATE Validated Data Files Insertion with the import tools Optional expert data validation Internal ESPON DB External ESPON DB OUTPUT DATA Extraction with the import tools ESPON Projects External World

49 The Data In-Flow of the ESPON 2013 DataBase Data inputs coming from other ESPON Projects will be composed of two files: 1) One data file containing the supplied dataset e.g one Excel file with one data sheet (NUTS version, Date, Indicator) + one values sheet (Code, Value) 2) An INSPIRE Compliant XML metadata file Metadata keep track of the source, quality and methods used and will be augmented with data completeness statistics First metadata editor: geoportal.eu/inspireeditor/ A more adapted version (enriched with data dictionaries) will be built using open source INSPIRE metadata compliant editor CatMDEdit: Work on Metadata in collaboration with UL, IGEAT, UAB and RIATE

50 The Data Out-Flow of the ESPON 2013 DataBase Data outputs for other ESPON Projects or/and the External World Data files (diffusion tables) available through the Web interface of the External ESPON 2013 Database In addition, at each territorial level, for each country, the completeness of data indicators will be summarized in a synthetic table

51 Challenge 9 : Data model and integration Objectives: To build an operational software architecture that ensures the importation, the integration, and the exportation of set of Indicators (data) Situation: ESPON DB 2006, Recommendations for a Long-Term DataBase in the ESPON Project 3.2 Strategy: 1) A model that supports the heterogeneity and evoluvity of data 2) Implementation Tools: open source and compatible with standards 3) Step-by-step validation: a progressive integration of the results of the other challenges Coord: LIG (Grenoble Computer Science Laboratory)

52 ESPON 2013 DataBase: Architecture t Relies on three connected pools (set of tools) Importation Pool: receives data from various sources (indicator providers) Integration Pool: integrates harmonised and well-documented data. When needed, d quality controls and missing i values estimations are performed Exportation Pool: delivers data to other ESPON Projects and to the External World as well

53 ESPON 2013 DataBase Version 1.0 EU15, EU27, etc. Indicator types: population, unemployment, etc. Indicator values: the actual data Bulgaria, Paris, etc. Link withhigh detail NUTS0 geometry

54 ESPON 2013 DataBase Version 1.0 A very first version of the ESPON 2013 DataBase Development has just started Implementation using PostGreSQL 8.3 1) Implementation of the DataBase Schema 2) Import Tool: a semi-automatic data acquisition process which has led to a first normalized input format 3) Export Tool: a basic interface to export data sets in the shape of Excel file C l d f d d f S 2003 d Content: a limited set of indicators, providers for NUTS 2003 and NUTS 2006 (from level NUTS 0 to level NUTS 3)

55 ESPON 2013 DataBase Version 2.0 To be delivered in February 2009 (First Interim Report) ESPON 2013 DataBase Version 2.0 1) Import Tool completed with a first version of Metadata editor 2) A first version of the Internal Database A complex software structure relying on PostgreSQL/PostGIS DBMS CatM Integrating Geographic and Thematic Ontologies + Harmonization and Estimation Methods (see ESPON 3.2 Project, Long Term Data Base) 3) The External DataBase (Export Tool) will be made available 3) The External DataBase (Export Tool) will be made available through a Java Web server

56 ESPON 2013 DataBase Version 2.x In more advanced stages of the Project: More harmonized data Importation and Exportation of Data under various formats (Excel, text, GIS standards, etc.) Metadata (enriched INSPIRE compliant XML file)