EDAMIS The naming convention For 1. Datasets, 2. Dataset occurrences This document covers the 2 possible levels of names related to datasets: 1. Dataset: mainly used in EDAMIS components (Dataset inventory, ewp, ewa) as well as in GESMES and SDMX-ML 2. Dataset occurrence: - a dataset occurrence identifies a dataset for one country and one period (or time series or sequence), - when used by data senders to identify files to be sent to Eurostat, it allows an automatic transmission or at least an assistance to the construction of the envelope (same approach as when GESMES is used) Examples in this document are based on existing dataset id's. Due to new regulations, directives, etc. these dataset id s can change. These changes do not affect the examples as these only intend to support the understanding of the presented definitions. Document prepared by Eurostat Unit B5 Version 5.01, Date: 4 January 2010 Page 1 of 8
Version Nr. Version Date Author 4 1 February 2006 H. Cloodt V. Tronet Version Control Description Of the modification A clear split into two different documents between the dataset occurrence naming convention which should be used by data senders and the data files delivered naming convention which is used by edamis at the delivery of data. This document is now in line with this current situation and is for Eurostat internal use. Modified Page(s) and/or Chapter(s) All Further improvements: - 1: cosmetic changes and new example 2-2 dataset occurrence naming convention PERIOD field, change of description for others replaced by for other than those and cosmetic changes + new examples + add optional field and format to the name + special case related to different periods for data and transmission + Special case related to the country code used when data is sent by Eurostat to a partner country. - 3 data file naming convention examples updated and action code is no more optional - 4 the naming convention at a glance, correction in the Dataset naming convention of Periodicity ID for data into Periodicity or periodicities, deleted Transmission part in the example of the dataset occurrence naming convention. Furthermore new examples and change in the structure of the dataset occurrence and data file delivered 4.01 21/12/2006 V. Tronet In Annex-1 (code for International Organisations) changed the yellow background from code "4A" to code "4D" because "4D" is the code to be used for Eurostat or more globally the European Commission. 4.02 04/07/2008 H. Cloodt In 3 a remark has been introduced indicating the foreseen phase out of the action code field and explaining how to use the filename of data files delivered by edamis 5.00 25/09/2009 H. CLoodt - definition of dataset occurrence updated to cover outgoing datasets - data file delivered definition: action code removed -naming convention at a glance updated 5.01 04/01/2010 H. Cloodt Some corrections carried out after having received feedback from the development team Page 7 Page 4, 5, 6 Page 4, 6, 7 Version 5.01, Date: 4 January 2010 Page 2 of 8
The naming convention described below is used by all the Data Transmission Tools and Services. 1. DATASET NAMING CONVENTION: Definition Dataset : concept of one or several statistical tables (having one single data structure), with a specific periodicity and to which is usually associated a maximum delay for transmission. Identification rule: DATASET ID = (DOMAIN ID) + _ + (DATASET STRUCTURE ID) + _ + (PERIODICITY or PERIODICITIES) Field: Length: Description/Remark: DOMAIN ID: 1..8 Identifies the statistical domain (group of datasets closely linked together) DATASET STRUCTURE ID: 1..7 Identifies the Dataset Structure (associated to one or several statistical tables). If no Dataset Structure ID is defined, the periodicity id for data will be used as default value. If 2 positions are used for the Periodicity or periodicities field, then the Dataset structure ID should be on 6 positions maximum. PERIODICITY or PERIODICITIES: 1..2 The first position, identifies the periodicity of the data to be reported: A for Annual 2 to 9 for every 2 to 9 years S for Semester Q for Quarterly M for Monthly W for Weekly D for Daily O for Other (periodicity) N for Non periodic (e.g. Sequential) The second position identifies the periodicity of transmission. It has to be provided only if the periodicity for transmission is different from the periodicity of the data. It uses the same code as the first position. Composition constraints and limitations for fields: Only the uppercase characters from A to Z and the figures from 0 to 9 are allowed For DOMAIN ID, the first character must be alphabetic The _ (underscore) character can only be used as a field separator. Based on the above the theoretical maximum length for a dataset ID is 8+1+7+1+1 or 8+1+6+1+2= 18 (in compliance with the maximum length defined in GESMES). EXAMPLE1: Statistical domain: RAIL TRANSPORT, Dataset structure: Annex E (tables E1 and E2 quarterly goods) Periodicity for data and transmission: Quarterly (Q) If RAIL is chosen as identification for the statistical domain, then the dataset id to be used is: RAIL_E_Q. EXAMPLE2: Statistical domain: Structural Business Statistics - Annex 4 - Construction, Dataset structure: 4L2 (Purchases of Energy Products (year 2002 onwards)) Periodicity for data: Annual (A) Periodicity for transmission: Every 2 years (2) If SBSCON is chosen as identification for the statistical domain, then the dataset id to be used is: SBSCON_4L2_A2. Version 5.01, Date: 4 January 2010 Page 3 of 8
2. DATASET OCCURRENCE NAMING CONVENTION: Definition Dataset occurrence : An occurrence of a dataset, for 1 sending country (or 1 international organisation) and 1 receiving country (or 1international organisation), for 1 period (or time series or sequence). Identification rule: DATASET OCCURRENCE ID = (DATASET ID) + _ + (FROM) + _ + (YEAR) + _ + (PERIOD) + _ + (TO) [+ _ + (optional field(s))] +. + (FORMAT) Field: Length: Description/Remark: DATASET ID: See above See above FROM: 2 The country code of the country to which the data providing organisation belongs or in case of an international organisation the code as listed in Annex 1. ISO country code should be used except for: GB replaced by UK, GR replaced by EL For international organisations use codes listed in Annex-1 except for: 4D replaced by EU. YEAR: 4 Format: YYYY, identifies the reporting year. If the dataset is non periodic then 0000 or reporting year PERIOD: 4 Identifies the reporting period: 0000 for Annual or for every 2 to 9 years, 0001 to 9999 for other than those. TO: 2 The country code of the country to which the data receiving organisation belongs or in case of an international organisation the code as listed in Annex 1 ISO country code should be used except for: GB replaced by UK, GR replaced by EL For international organisations use codes listed in Annex-1 except for: 4D replaced by EU. Optional field(s) 1 220 Though not recommended, optional information given by the data sender (not taken into account at all by ESTAT data transmission tools) FORMAT 3 Examples: "XML"; GES : GESMES, CSV, FLR : Fixed length Records,. Composition constraints and limitations for fields: - YEAR and PERIOD : If several years/periods are covered by a dataset occurrence (case of time series), then only the last year/period should be specified, - YEAR and PERIOD : the year/period of the data should be specified except if the periodicity for transmission is different from the periodicity of the data (the year/period of transmission should be specified in that case), - Optional field(s): Only characters A to Z, the figures from 0 to 9 and _ are allowed. EXAMPLE INCOMING DATASETS (received by Eurostat): EXAMPLE1 : For Dataset RAIL_E_Q Country: United Kingdom (UK) Year: 2003 Quarter: 2 Format: CSV (Comma Separated Values) The Dataset occurrence id to be used is: RAIL_E_Q_UK_2003_0002_EU.CSV EXAMPLE2: For dataset MILK_TABLEA_M Country: Greece (EL) Year: 2002 Month: 12 Format: SDMX-ML The Dataset occurrence id to be used is: MILK_TABLEA_M_EL_2002_0012_EU.XML Version 5.01, Date: 4 January 2010 Page 4 of 8
EXAMPLE3: For Dataset ESANAAGR_0109_A Country: France (FR) Year: 2001 Optional information: TEMP (Temporary) Format: Excel The Dataset occurrence id to be used is: ESANAAGR_0109_A_FR_2001_0000_EU_TEMP.XLS EXAMPLE4: Name with transmission frequency indication For Dataset SBSCON_4L2_A2 Country: Italy (IT) Year of the data: 2005 Format: GESMES The Dataset occurrence id to be used is: SBSCON_4L2_A2_IT_2005_0000_EU.GES EXAMPLE OUTGOING DATASETS (sent by Eurostat): For Dataset EGROUT_ASSETS_A Receiving country: United Kingdom (UK) Year: 2008 Format: CSV (Comma Separated Values) The Dataset occurrence id to be used is: EGROUT_ASSETS_A_EU_2008_0000_UK.CSV Version 5.01, Date: 4 January 2010 Page 5 of 8
3. The naming convention at a glance: DATASET Naming Convention DOMAIN ID _ DATASET STRUCTURE ID _ PERIODICITY OR PERIODICITIES Examples: RAIL_E_Q SBSCON_4L2_A2 DATASET OCCURRENCE Naming Convention DATASET ID _ FROM _ YEAR _ PERIOD _ TO [_ Option]. FORMAT Examples: RAIL_E_Q_UK_2003_0002_EU.CSV MILK_TABLEA_M_EL_2002_0012_EU.XML ESANAAGR_0109_A_FR_2001_0000_EU_TEMP.XLS SBSCON_4L2_A2_IT_2005_0000_EU.GES EGROUT_ASSETS_A_EU_2008_0000_UK.CSV Version 5.01, Date: 4 January 2010 Page 6 of 8
Annex 1 : Code to be used for international organisations (in the country code field) The codes highlighted in yellow are related to the main partners of Eurostat. Code List CL_AREA_EE (extract for international organisations) 1A 1B 1C 1D 1E 1F 1G 1H 1J 1K 1L 1M 1N 1O 1P 1Q 1R 1S 1T 1Z 4A 4B 4C 4D 4E 4F 4G 4H 4J 4K 4L 4M 4N 4P 4Q 4Z 5A 5B 5C 5D 5E International organisations code list (extract from ECB metadata Area code list) International organisations UN organisations International Monetary Fund World Trade Organisation International Bank for Reconstruction and Development International Development Association Other UN Organisations (includes 1H, 1J-1T) UNESCO (United Nations Educational, Scientific and Cultural Organisation) FAO (Food and Agriculture Organisation) WHO (World Health Organisation) IFAD (International Fund for Agricultural Development) IFC (International Finance Corporation) MIGA (Multilateral Investment Guarantee Agency) UNICEF (United Nations Children Fund) UNHCR (United Nations High Commissioner for Refugees) UNRWA (United Nations Relief and Works Agency for Palestine) IAEA (International Atomic Energy Agency) ILO (International Labour Organisation) ITU (International Telecommunication Union) Rest of UN Organisations n.i.e. European Community Institutions, Organs and Organisms EMS (European Monetary System) European Investment Bank European Commission European Development Fund European Central Bank EIF (European Investment Fund) European Community of Steel and Coal Other EC Institutions, Organs and Organisms covered by General budget European Parliament European Council Court of Justice Court of Auditors Economic and Social Committee Committee of Regions Other European Community Institutions, Organs and Organisms Organisation for Economic Cooperation and Development Bank for International Settlements Inter-American Development Bank African Development Bank Asian Development Bank Version 5.01, Date: 4 January 2010 Page 7 of 8
5F 5G 5H 5J 5K 5L 5M 5N 5P 5Q 5R 5S 5T 5U 5V 5Z 6A 6B 6C 6D 6E 6F 6G 6H 6J 6K 6L 6M 6N 6O 6P 6Q 6Y 6Z 7Z European Bank for Reconstruction and Development IIC (Inter-American Investment Corporation) NIB (Nordic Investment Bank) IBEC (International Bank for Economic Co-operation) IIB (International Investment Bank) CDB (Caribbean Development Bank) AMF (Arab Monetary Fund) BADEA (Banque arabe pour le développement économique en Afrique) CASDB (Central African States Development Bank) African Development Fund Asian Development Fund Fonds spécial unifié de développement CABEI (Central American Bank for Economic Integration) ADC (Andean Development Corporation) Other International Organisations (financial institutions) Other International Financial Organisations n.i.e. Other International Organisations (non-financial institutions) NATO (North Atlantic Treaty Organisation) Council of Europe ICRC (International Committee of the Red Cross) ESA (European Space Agency) EPO (European Patent Office) EUROCONTROL (European Organisation for the Safety of Air Navigation) EUTELSAT (European Telecommunications Satellite Organisation) INTELSAT (International Telecommunications Satellite Organisation) EBU/UER (European Broadcasting Union/Union européenne de radio-télévision) EUMETSAT (European Organisation for the Exploitation of Meteorological Satellites) ESO (European Southern Observatory) ECMWF (European Centre for Medium-Range Weather Forecasts) EMBL (European Molecular Biology Laboratory) CERN (European Organisation for Nuclear Research) IOM (International Organisation for Migration) Other International Organisations (financial institutions) Other International Non-Financial Organisations n.i.e. International Organisations excluding Eur.Community Institutions (4A) Version 5.01, Date: 4 January 2010 Page 8 of 8