SDMX in the UNIDO Statistical Production Process

Transcription

1 1 SDMX in the UNIDO Statistical Production Process UNIDO OECD/UNECE Expert Group on SDMX (Geneva 8-9 March 2010)

2 2 Outline Introduction and Overview UNIDO statistical production process and GSBPM To start with: consuming the OECD.Stat Web Service Exchanging data and metadata with SDMX-ML Collaborative software development Summary and Outlook

3 3 About UNIDO UNIDO was set up in 1966 Became a specialized agency of the UN in 1985 Promote industrialization throughout the developing world 173 Member States (as of 11 December 2008) Headquarters in Vienna Represented in 35 developing countries

4 4 About Statistics in UNIDO Service Module Industrial Governance and Statistics : monitor, benchmark and analyse the industrial performance and capabilities formulate, implement and monitor strategies, policies and programmes to improve the contribution of industry to productivity growth and the achievement of the UN Millennium Development Goals (MDGs) Building capabilities in industrial statistics - providing technical assistance to: Introduce best practice statistical methodologies and software systems Enhance the quality and consistency of the industrial statistics databases

5 5 Management Processes Quality Management INDSTAT Data Collection INDSTAT Data Transformation INDSTAT Data Analysis INDSTAT Data Dissemination INDSTAT Metadata Management Sales and marketing Support Processes User support (stat.info) Methodology development Software development

6 6 NSO NSONSO A.1 Structural Metadata Maintenance B.1 INDSTAT Data and metadata Transformation D.1 Yearbook production OECD UNSD World Bank IMF A.2 INDSTAT data and metadata collection B.2 Index of industrial production B.3 MVA/GDP B.4 CPI and Exchange rates UNSD SQL Data & Metadata B.5 Population B.6 Trade data COM- TRADE D.2 INDSTAT 2 CD production D.3 INDSTAT 4 CD production D.4 IDSB CD production D.5 WEB: Country brief production D.6 UNDATA

7 7 NSO NSONSO A.1 Structural Metadata Maintenance B.1 INDSTAT Data and metadata Transformation D.1 Yearbook production OECD UNSD World Bank IMF A.2 INDSTAT data and metadata collection B.2 Index of industrial production B.3 MVA/GDP B.4 CPI and Exchange rates UNSD SQL Data & Metadata B.5 Population B.6 Trade data COM- TRADE D.2 INDSTAT 2 CD production D.3 INDSTAT 4 CD production D.4 IDSB CD production D.5 WEB: Country brief production D.6 UNDATA

8 8 NSO NSONSO A.1 Structural Metadata Maintenance B.1 INDSTAT Data and metadata Transformation D.1 Yearbook production OECD UNSD World Bank IMF A.2 INDSTAT data and metadata collection B.2 Index of industrial production B.3 MVA/GDP B.4 CPI and Exchange rates UNSD SQL Data & Metadata B.5 Population B.6 Trade data COM- TRADE D.2 INDSTAT 2 CD production D.3 INDSTAT 4 CD production D.4 IDSB CD production D.5 WEB: Country brief production D.6 UNDATA

9 9 NSO NSONSO A.1 Structural Metadata Maintenance B.1 INDSTAT Data and metadata Transformation D.1 Yearbook production OECD UNSD World Bank IMF SDMX-ML A.2 INDSTAT data and metadata collection B.2 Index of industrial production B.3 MVA/GDP B.4 CPI and Exchange rates UNSD SQL Data & Metadata B.5 Population B.6 Trade data COM- TRADE D.2 INDSTAT 2 CD production D.3 INDSTAT 4 CD production D.4 IDSB CD production D.5 WEB: Country brief production D.6 UNDATA

10 10 Data cubes (the data we are interested in) Data set: SSIS_BSC Structural Business Statistics (Business by size class) Time series - annual data - all available years Dimensions: COU - Country: AUS, AUT, BEL, DEU, etc. SRC - Source: BSC - Business by Size Class SCL - Size class: Total ISIC3 ISIC Revision 3 - all ISICs at 2-, 3- and 4-digit level.

11 11 Data cube summary (cont) Dimensions (cont): VAR - Variable: ENTR - Number of enterprises ESTB - Number of establishments WASE - Wages & Salaries of employees EMPE - Number of employees EMPN - Total employment (number engaged) EMPF - Number of female employees VAFC - Value added at factor costs VALU - Value added PROD - Production GITG - Gross investment

12 12 OECD Import the process

13 13 OECD Import Code Mapping Variables Countries

14 14 OECD Import INDSTAT database

15 15 Web Services Are fundamental building blocks of distributed computing over the Internet Provide the means for software to connect to other software applications Use ubiquitos web protocols and formats: HTTP, XML, SOAP WSDL Service provider and service consumer are implementation independent Keep the stove pipes apart SDMX benefits from having a common information model: we can base WS on it

16 16 OECD.Stat Web Service Gives access to.stat data (Almost) fully SDMX 2.0 compliant Some limitations apply: Only annual, bi-annual, etc. data OK Each data set has own DSD OK Each query defines a separate data cube OK features Streaming, compression time stamps, incremental updates metadata structure definitions and reference metadata

17 17 OECD.Stat WS functions used (Data) Authenticate returns a session ID used for all subsequent WS operations the same session ID can be reused for several SDMX requests within a certain timeout without authentication retrieval of limited data set possible GetDataStructureDefinition to retrieve the Data Structure Definition, Code lists and Concepts related to the data set SSIS_BSC GetGenericData to retrieve data in generic format from the data set SSIS_BSC specify country and a list of variables in the query time series - annual data for all available years

18 18 Data Structure Definition Query Example Select data set: SSIS-BSC <message:query> <KeyFamilyWhere> <Or> <KeyFamily>SSIS_BSC</KeyFamily> </Or> </KeyFamilyWhere> </message:query>

19 19 Generic Data Query Example Select data set SSIS-BSC Select country: Austria Select variables: ENTR, ESTB, WASE, etc. Wild card select all ISIC codes by omitting the dimension Set time format - annual Set time interval

20 20 Client Application (Data)

21 21 Reference Metadata

22 22 Reference Metadata

23 23 OECD.Stat functions used (Reference Metadata) Authenticate GetMetadata to retrieve reference metadata in generic format from the data set SSIS_BSC only metadata at level data set and at level country are available specify country (dimension COU)

24 24 Metadata query: dimension member Select data set: SSIS-BSC Query a specific dimension: country <MetadataWhere> <And> <DataSet>SSIS_BSC</DataSet> <Dimension id = "COU">AUT</Dimension> </And> </MetadataWhere>

25 25 Client Application (Reference Metadata)

26 26 Collaborative development A low budget, volunteer project A team spread through multiple time zones and geographical locations Collaboration tools for virtual teams: A Wiki was set up at wikispaces.com, to build SDMX knowledge, to specify the basic requirements and the desired output: Publicly available at CloudShare - to share copies of the IT environment - for Demos, Proofs-of-Concept, Training, or other applications Google Docs at for collaborative editing of documents, presentations, etc.

27 27 Collaborative development: Wiki

28 28 Collaborative development: CloudShare

29 29 NSO NSONSO OECD UNSD World Bank SDMX-ML SDMX-ML A.1 Structural Metadata Maintenance A.2 INDSTAT data and metadata collection B.2 Index of industrial production B.3 MVA/GDP B.1 INDSTAT Data and metadata Transformation SQL Data & Metadata B.5 Population B.6 Trade data SDMX-ML D.1 Yearbook production D.2 INDSTAT 2 CD production SDMX-ML D.3 INDSTAT 4 CD production SDMX-ML D.4 IDSB CD production SDMX-ML D.5 WEB: Country brief production SDMX-ML IMF SDMX-ML B.4 CPI and Exchange rates UNSD SDMX-ML COM- TRADE D.6 UNDATA SDMX-ML

30 30 THE END

31 31 Examples

32 32 Questionnaire

33 33 OECD Questionnaire